FlexInference

Search FlexInference docs

Keyword search over the FlexInference documentation (quickstart, authentication, deadline routing, supported models, SDK usage, and the four API endpoints). Use this for 'how do I...' questions, e.g. how to pass start_within through the official OpenAI SDK, or which models support the flex race.

Look up a FlexInference error code

Look up what a FlexInference error code means and how to fix it (status, type, full message, and a docs link). Pass the exact `code` from an error response's `error.code` field, e.g. "no_byok_key". Omit `code` to list every known code.

Run the two-request FlexInference demo

Run the homepage demo programmatically: one prompt is sent twice on the same model, once through the flex race and once as a plain default request, and the answer carries both outputs plus the true billed comparison (cost per lane and time to first token). One run per IP address, from a small shared daily budget; a second call answers demo_already_used.

Price a usage export against FlexInference

Paste the text of a usage CSV export (OpenAI platform, Anthropic console, or OpenRouter) and get back what those rows would have cost through FlexInference: total cost, savings at each model's measured realized rate, and the time-to-first-token trade, per model and in total. The export is priced and dropped, never stored. This tool takes exports up to about 1 MB; for a bigger file, POST it as text/csv to https://www.flexinference.com/api/compare.

Get usage and savings summary

This organization's flex-routed usage and savings: total requests, what was billed, what standard-tier pricing would have cost, and the difference, broken down by model. BYOK serving is free and unlimited, so there is no allowance to report; a managed organization also sees its prepaid wallet balance. The savings breakdown is scoped to requests that ran the flex race - only those save money vs. standard.

List FlexInference API keys

List this organization's FlexInference API keys: name, last 4 characters, creation time, and a truncated id (never the full key - it is only ever shown once, at creation). Keys are created and revoked in the FlexInference dashboard, not through this server.

Check BYOK key status

Check which providers (OpenAI, Gemini, Anthropic, Bedrock, Vertex) have a BYOK key configured for this organization. Returns yes/no per provider only - never the key itself, and there is no tool to set one (add or change a provider key in the dashboard).

Check billing status

Read this organization's billing status. BYOK serving is free and unlimited, so for a BYOK organization this reports only whether a payment method is on file (card brand and last 4 only - never a full card number). For a managed organization: its prepaid wallet balance (cash and promotional credit) and whether that balance is low or exhausted. Read-only - there is no tool to add or change a card or wallet balance (use the dashboard Billing page).