Skip to content
Hosted LLM Gateway · GDPR & KVKK ready

The control plane in front ofyour LLM traffic, every model call, your inference bill, your audit trail

Reach dozens of models through one OpenAI-compatible endpoint. Change routing, PII redaction, budgets and compliance without touching application code — the decision is made inside the gateway, in well under a millisecond.

  • No provider keys to manage
  • Policy changes need no deploy
  • A full decision trace per request
Request a demoTry the policy engine

Live request flow

One request, six stages — and what it looks like coming out

req_01JQ8Z3K7M2N4P6R8T0V2X4Y6A
Elapsed0µs
  1. App
  2. Auth
  3. Signals
  4. Policy
  5. Route
  6. Response
Promptbefore it reaches a provider

10000000146 TCKN'li müşterinin limitini artır

Your application

OpenAI SDK · only the base_url changes

POST /v1/chat/completions{"model": "combo/production-chat","messages": [{ "role": "user", … }],"stream": true}

Tap a stage — the walk pauses and stays there

Provider surface

One integration, every provider

Your application only ever talks to Modelion. Which provider runs behind it is a policy decision — and changing that decision doesn't ship code.

  • OpenAIOpenAI-wire
  • AnthropicNative adapter
  • AWS BedrockNative adapter
  • Google Vertex AINative adapter
  • Azure OpenAIOpenAI-wire
  • Together AIOpenAI-wire
  • Fireworks AIOpenAI-wire
  • GroqOpenAI-wire
  • DeepInfraOpenAI-wire
  • vLLMYour own infrastructure

Adding another OpenAI-wire provider is a catalog operation, not a code change.

Measured, not rounded

The control plane in numbers

These figures come out of the product's architectural decisions rather than a marketing round-up.

222µs

p99 policy decision

Rego compiles to WASM and evaluates inside the gateway process. No network hop.

5

decision axes

route · constrain · deny · redact · cache — all in one engine.

3

data regions

tr-west-1 · eu-east-1 · us-east-1. Residency is enforced by policy.

0

customer API keys

Modelion runs the provider relationships; key custody isn't your problem.

Why a gateway

When compliance lives in if blocks

Every team that calls providers directly rediscovers the same five problems independently.

Without a gateway

  • Every policy change is a deployChanging one PII rule means waking the release pipeline — two weeks, typically.
  • Rules scattered through the codebaseThere is no single place that answers why a given request went to a given model.
  • Provider keys are your liabilityRotation, rate limits and leak risk all sit with the engineering team.
  • Cost is invisible per requestThe bill arrives as one line at month end; which feature spent what stays a guess.
  • Trying a new model is a riskWith no safe way to compare in production, model migrations keep getting deferred.

With Modelion

  • Policy changes in minutesYou publish a YAML rule set; it reaches gateway pods as a signed bundle.
  • One engine holds the decisionsPlatform baseline, organization governance and custom rules merge in one pool.
  • Keys live in Modelion's poolAES-256-GCM encrypted, KMS-wrapped, decrypted only for the duration of a request.
  • Tokens, latency and cost per requestBroken down by virtual key, model and prompt version — and queryable.
  • Shadow first, enforce secondRun the rule in observe mode and measure divergence; enforce with a single call once agreement is good enough.
Capabilities

Five capabilities, one control plane

Each could have been a product on its own. Because they share one decision contract, together they do more.

Interactive · Policy engine

Make the decision yourself

The switches below are the real inputs of a real rule set. Change them and you see which rule wins, where the request goes and which obligations that creates.

Request context

Subscription tier
1,200
Data class

Decision

Decision time 201µs
Allow
Winning rule
default-allow-with-cache
Selected model
gpt-4o-mini
Region
tr-west-1
Semantic cache
Read and write
Matched and lost
None
Obligations
  • A decision digest is added to the response headers

No restrictive rule matched. The request goes to the default model pool and the semantic cache stays on.

The rule set being evaluated
apiVersion: modelion.ai/v1kind: PolicyRuleSetmetadata:  name: production-governance  class: compliancespec:  - name: regulatory-data-deny    when:      - field: request.headers.dataClass        operator: equals        value: regulatory    then:      effect: deny      statusCode: 403  - name: kvkk-pii-route-resident    when:      - field: signals.pii.detected        operator: is        value: true    then:      effect: route      route:        candidates: [gpt-4o-mini]        region: tr-west-1        fallbackToOriginal: false      obligations:        redactTypes: detected        auditTags: [kvkk_pii_detected]        cache: { mode: off }  - name: free-tier-constrain    when:      - field: principal.subscription.tier        operator: equals        value: free    then:      effect: constrain      constraints:        allowedModels: [llama-3.3-70b, gpt-4o-mini]        maxOutputTokens: 500      obligations:        cache: { mode: lookup_only }  - name: long-context-upgrade    when:      - field: request.prompt.tokens        operator: greaterThan        value: 4000    then:      effect: route      route:        candidates: [claude-sonnet-4, gpt-4o]  - name: default-allow-with-cache    then:      effect: allow      obligations:        cache:          mode: read_write          scope: org          threshold: 0.85          ttlSeconds: 3600

This demo runs in your browser and makes no gateway call. The engine itself evaluates the same rule contract on WASM in production.

Request lifecycle

The 8 stages a request passes through

The order is not arbitrary. Prompt hydration, for instance, runs before policy evaluation: PII arrives inside the variables, so filling the template afterwards would mean scanning the wrong string.

  1. 01

    Auth & quota

    The virtual key is verified, organization and region context resolved, rate limits applied.

    mk_live_… · X-Modelion-Organization
  2. 02

    Signal hydration (PIP)

    Guardrail scanners, budget state and provider health are written into the input the policy will see.

    signals.pii · signals.injection · runtime.health
  3. 03

    Prompt hydration

    If the request arrived with a prompt_id and variables, the template is filled here — before policy, because the PII is inside those variables.

    prompt registry · signed bundle
  4. 04

    Policy decision (PDP)

    Platform baseline, organization governance and custom rules evaluate into one pool; constraints intersect, obligations union.

    Rego → WASM · p99 222µs
  5. 05

    Enforcement (PEP)

    This is where the decision becomes action: deny, mask the prompt, narrow the model pool, flag the obligations.

    deny · redact · constrain
  6. 06

    Semantic cache

    If policy allows it, the answer to a similar prompt is served. When PII is detected, cache writes are switched off.

    pgvector · org scope · threshold 0.85
  7. 07

    Routing & failover

    The combo resolves and candidates are tried in order; failover is possible right up to the first chunk. The circuit breaker takes an unhealthy route out of rotation.

    combo/production-chat · 20 candidates
  8. 08

    Budget, execution & telemetry

    Spend is reserved, the request goes upstream, and the decision trace plus usage record are written.

    ledger · decision_traces · Modelion-Decision
Interactive · Guardrails

The layer that reads the prompt before policy does

Whatever you type below is scanned in your browser, with the same kind of patterns the gateway uses. The resulting signals become policy input — which is what makes a rule like "if there's a national ID, pin it to Turkey" possible at all.

Prompt

155
Sample payloads

What was detected

Hi, can you raise the credit limit for the customer with national ID 10000000146 to 25,000? Contact: ayse.yilmaz@examplebank.com, card 4111 1111 1111 1111.

  • Turkish national ID1
  • Card number1
  • Email1
Signals handed to policyinput.signals
{
  "pii": { "detected": true, "types": ["tckn", "email", "creditCard"], "available": true },
  "secrets": { "detected": false, "available": true },
  "injection": { "detected": false, "score": 0.00, "available": true }
}

The masked version

What reaches the provider once a redactTypes: detected obligation applies

Hi, can you raise the credit limit for the customer with national ID [TCKN_TR] to 25,000? Contact: [EMAIL], card [CARD].

Scanning happens entirely in your browser; nothing you type is sent anywhere.

Interactive · Routing

The provider goes down, the request doesn't

A combo is an ordered list of candidates. Take one of the providers below out of service and watch how far down the chain the request travels.

combo/production-chat

Click to take a provider out of service

Result

Send a request →

Per-step conditions

  • Token range — short requests drop to the cheap model
  • Route tag — the Modelion-Route-Tag header
  • Streaming — the streaming path can branch separately
  • Content keyword — "invoice" → the finance model

Circuit breaker

Past a consecutive-failure threshold the route drops out of rotation for a while. Health reaches policy through input.runtime — but if everything is unhealthy no bypass happens: failing open must not widen the compliance boundary.

Observability

The answer to "why did this model produce this answer?"

A decision trace keeps not only the winning rule but the ones that matched and lost. In an audit, that is the actual question.

Requests this month

4.8M

sliceable by virtual key
p99 latency

812 ms

policy + routing + provider
Cache hit rate

31.4%

semantic, org-scoped
Estimated cost

$6,240

token-based, live

Request volume and p99 latency

Last 24 hours

Live
Request volumep99 latency

Model mix

  • gpt-4o-mini42%
  • llama-3.3-70b27%
  • claude-sonnet-419%
  • gpt-4o12%

Decision traces

TimeWinning ruleEffectModelLatencyCost
14:22:07kvkk-pii-route-residentroutegpt-4o-mini812 ms$0.0021
14:22:07default-allow-with-cachecache hit—41 ms$0.0000
14:22:06free-tier-constrainconstrainllama-3.3-70b394 ms$0.0003
14:22:05regulatory-data-denydeny—6 ms$0.0000
14:22:04long-context-upgraderouteclaude-sonnet-41 284 ms$0.0184

The Modelion-Decision header

Every response carries a decision digest under 512 bytes. The full trace is fetched by requestId.

Interactive · Cost

Same traffic, smaller bill

There are two levers: similar requests returning from the semantic cache, and low-risk traffic dropping to a cheaper model. Policy decides both — not code.

2M
1,400
%30
%45

This is an order-of-magnitude estimate using example unit costs close to public list prices. Real savings depend on the shape of your traffic; talk to us to run it against your own numbers.

Everything on one premium model

$12,250

With policy-driven routing

$4,960

Monthly saving

$7,290

−60%

Per year: $87,476

Where it comes from

  • Requests served from cache600K · $13
  • Traffic on the cheap model630K · $232
  • Traffic staying on premium770K · $4,716
Developer experience

The only thing that changes in your code is base_url

Modelion speaks OpenAI's wire format. Your streaming, tool-call and embedding calls keep working as they are; which provider sits behind them is a policy decision.

 1from openai import OpenAI 2 3client = OpenAI( 4    # The only line that changes. 5    base_url="https://api.modelion.ai/v1", 6    api_key="mk_live_...", 7) 8 9response = client.chat.completions.create(10    # A combo, not a model: the candidate chain lives in the console.11    model="combo/production-chat",12    messages=[{"role": "user", "content": "Summarise this invoice."}],13    stream=True,14)1516for chunk in response:17    print(chunk.choices[0].delta.content or "", end="")

The decision digest that comes back with the response

Modelion-Decision: v1;
  rule=kvkk-pii-route-resident;
  effect=route;
  model=gpt-4o-mini;
  region=tr-west-1;
  obligations=redact,no-fallback;
  cache=off;
  eval=181us;
  requestId=req_01JQ8Z3K7M2N4P6R8T0V2X4Y6A

Every sample uses your mk_live_… virtual key. A key is bound to one organization, one region and one budget.

Full wire-format reference
Use cases

One engine, three different problems

Fintech · KVKK

A request that sees a national ID stays in Turkey

The guardrail detects the ID, policy pins the request to a model in tr-west-1, masks the prompt and switches cache writes off. The fallback chain is deliberately disabled — in a compliance path, failover is a vulnerability.

to change the policy
5 min
code deploys
0
B2B SaaS · Cost

Tiered pricing enforced at the gateway

The free tier is constrained to the cheap model pool, the pro tier is routed by token range, and similar requests come back from the semantic cache. Changing a price tier no longer touches the backend.

of traffic cacheable
40%+
tiers, one rule set
4
Platform teams · Migration

Measure a new model in production, without the risk

Run the rule in shadow mode: the decision is produced but not enforced. When the divergence report shows agreement with current behaviour, enforce it with a single call.

observe first
shadow
call to enforce
1
FAQ

You're probably about to ask these

No. Modelion is hosted-only: it runs every provider relationship and holds every credential. That is a deliberate simplification — rotation, rate limits and leak exposure stop being your problem, and on our side there is a single encrypted credential pool to manage.

The policy decision happens inside the gateway process, on WASM: p50 154µs, p99 222µs. Next to the network round trip to the provider that is measurable but not perceptible. On a semantic cache hit, total latency goes down.

Rule sets are written as YAML and published from the console; for advanced needs you can upload Rego modules directly. Ready-made compliance templates were deliberately removed: every template was right for one tenant and quietly wrong for the next.

That is exactly what shadow mode is for: the rule evaluates, the decision is recorded, nothing is enforced. You enforce after the divergence report shows agreement with current behaviour. Publishing also measures what the bundle costs to evaluate, and refuses one that would exhaust the per-request fuel budget.

Your organization is bound to a home region: tr-west-1, eu-east-1 or us-east-1. You can enforce residency at the policy level — forbidding, say, a request with a detected national ID from leaving the region.

Yes. Whatever the upstream, the client sees an OpenAI-shaped stream; Anthropic's multi-event SSE framing is mapped onto the same ChatChunk stream. Failover stays possible up to the first chunk.

On the application side, a base_url and a key. The real work is porting the policies and validating them in shadow mode — typically a few days. Our solution architects can walk through a migration plan with you.

On-prem licensing is on the table for enterprise plans, and registering your own vLLM cluster as a catalog provider is a standard setup step. Talk to sales for the details.

Next step

Put your traffic behind the gateway once

We start with a 30-minute session where we run your own rule set in shadow mode. You leave with a divergence report measured against your real traffic.

Request a demoCompare plans
  • No credit card needed
  • Measured on your own traffic
  • Support in English and Turkish