Inside Vyom's routing decision: cost, capability, and failover

Hard-coding one vendor's SDK into your product makes three silent bets: that their pricing won't change, that their availability is your availability, and that the model that's best today will be best next quarter. All three bets lose eventually. Vyom exists to make them config instead of code.

The routing table

Every route declares a capability class (extraction, long-context reasoning, code generation, low-latency chat), an ordered candidate list, and constraints — max cost per 1K tokens, max p95 latency, data-residency rules. The router resolves each request against live provider health and current pricing, not a static preference.

• Failover is automatic and logged: if the primary times out, the request replays against the next candidate with the same contract
• Every completion is attributed: feature, team, model, token counts, cost — queryable SQL, not a vendor dashboard
• Routing changes are config deploys, reviewable in a pull request like any other change
• The gateway speaks an OpenRouter-compatible API, so adopting it is a base-URL change, not a refactor

What self-hosting buys here

Prompts and completions are some of the most sensitive traffic a company produces — they contain your product's internals and your customers' data. Vyom runs inside your network: providers see only what you route to them, and your usage data accumulates in your database as an asset, not in someone else's analytics.