Foundations first.
Runtime next.

OpenAI-compatible Chat Completions — multi-turn, multimodal content (text + base64 images), tool-calling, SSE streaming. Semantic reranking and embeddings as native endpoints.

Each model alias maps to a primary backend with an optional configurable fallback. A per-provider circuit-breaker (closed / open / half-open) trips on transient errors, and a smart router applies per-context defaults or header-driven overrides.

Every LLM call is recorded in an append-only journal — alias, effective provider, route, latency, tokens, HTTP code, role id — with 30-day retention, a 5M-event cap, and Prometheus export. Conversation content is never captured.

Endpoints are guarded by Bearer token with constant-time comparison and an optional loopback bypass keyed on the real TCP address. Business-context ids (feature, agent) are propagated into telemetry — never exposed in logs.

A native Rust supervisor runs inference processes directly — managing GPU, context window, slot parallelism and sampling per model instance, under strict execution-environment isolation and graceful shutdown.

Each inference instance exposes a real-time health endpoint (starting / ok / unhealthy) and Prometheus metrics (requests, latencies), with automatic supervision under a bounded restart budget and flapping detection.

Per-instance configuration is declarative TOML with environment overrides. Every request emits an async, non-blocking telemetry event (route, model, latency, HTTP code, role id) to the central journal — engine and server share the same core types and auth handshake.