Bind traffic to a decision, not to a model
A combo is an ordered set of candidates and per-step conditions. Your application says combo/production-chat; the gateway decides which model runs, in what order candidates are tried, and when a failing provider drops out.
- Failover chains and weighted splits
- Decision graphs up to 20 nodes
- Failover right up to the first chunk
20candidates · 10 items per chain
A model name shouldn't be an application detail
The moment gpt-4o is written into your code, changing that model becomes a deployment. A combo gives you the indirection: your application uses a stable name while you change the candidate list and conditions from the console. Policy can narrow that list further — the combo says what is possible, policy says what is permitted.
What it does
Failover chain
Candidates are tried in order. On streaming requests failover is possible until the first chunk arrives; after that the stream is never cut.
Weighted split
Split traffic 80/20 and compare two models under the same production load. Round-robin and weighted-random are both supported.
Decision graph
When a linear chain isn't enough: conditional branching, graphs up to 20 nodes, conditions per step.
Per-step conditions
Token range, route tag, streaming mode and content keywords. Requests mentioning "invoice" go to the finance model; short ones drop to the cheap model.
Circuit breaker
A route past its consecutive-failure threshold leaves rotation for a while. Health is made visible to policy through input.runtime.
Intersection with policy
A policy's allowedModels constraint intersects the combo's candidate list. A model present in the combo but not permitted by policy never runs.
The provider goes down, the request doesn't
A combo is an ordered list of candidates. Take one of the providers below out of service and watch how far down the chain the request travels.
combo/production-chatClick to take a provider out of service
Result
Send a request →
Per-step conditions
- Token range — short requests drop to the cheap model
- Route tag — the
Modelion-Route-Tagheader - Streaming — the streaming path can branch separately
- Content keyword — "invoice" → the finance model
Circuit breaker
Past a consecutive-failure threshold the route drops out of rotation for a while. Health reaches policy through input.runtime — but if everything is unhealthy no bypass happens: failing open must not widen the compliance boundary.
The demo on this page runs in your browser; no real gateway call is made.
How it works
Define the combo once and use its name in the application. Changes roll out without shipping code.
- 1
Define the combo
Order the candidates, set the per-step conditions, pick linear chain or decision graph.
combo/<key> - 2
Use its name in the app
Send
combo/production-chatas the model in the request body. A concrete model name is no longer an application detail.model: combo/production-chat - 3
Watch its health
Breaker state, per-candidate error rate and failover frequency, all on the observability dashboard.
runtime.health - 4
Shift it gradually
Add the new model at 10% weight, compare the metrics, raise the weight step by step.
weighted random
Technical summary
- Wire format
combo/<key>— in the model field of the request body- Modes
- Linear chain · decision graph
- Distribution
- Failover · round-robin · weighted random
- Items per chain
- 10 max
- Candidate ceiling
- 20 (fallback depth)
- Graph nodes
- 20 max
- Step conditions
- Token range · route tag · streaming mode · content keyword
- Failover window
- Until the first chunk, when streaming
If everything is broken, nothing is bypassed
The circuit breaker pulls an unhealthy route out of rotation, and policy can say "this provider is broken, take the request elsewhere". But if every candidate is unhealthy, no bypass happens: the gateway returns 503. Failing open never widens the compliance boundary — bending a residency rule to keep traffic flowing costs more than the outage would.
The rest of the control plane
This capability works on its own, but most of its value comes from sharing one decision contract with the others.
See it on your own traffic
In a 30-minute session we run your own rule set in shadow mode.