Cache, Stack, and Time: Three Ways to Slash Your LLM API Bill

I run five AI agents. They talk to LLMs all day, every day. For months I stared at my API bills and thought the problem was which model I was using or how much I was paying per token. I was wrong. The problem was one number: cache hit rate.

Not the per-token price. Not the model choice. Not even which provider I was calling. The single biggest lever on my LLM spending was whether the provider could reuse the work it already did on my previous request. And I had no visibility into it.

This post covers what I learned. Prompt caching, why small cache improvements cut your bill in half, how to install Bifrost for fleet-wide LLM logs (including subscriptions that give you zero visibility), a fix for OpenRouter’s cache fragmentation, and when to hit Chinese providers for maximum cache retention.

The number that matters more than price per token

Every major LLM provider has some form of prompt caching. When you send a request, the provider stores the processed tokens (the KV cache) for that prompt prefix. If your next request starts with the same prefix, the provider reuses the cached work instead of reprocessing from scratch. Cached tokens are billed at a fraction of the normal input rate, usually 10x to 20x cheaper.

Most agentic workloads resend 80-95% of the same tokens on every request. Your system prompt, tool definitions, conversation history, context documents. The only new tokens are the latest user message. If the provider caches that prefix, you pay almost nothing for the bulk of your input.

The providers know this. DeepSeek charges $0.0028 per million cached tokens versus $0.14 for fresh input on V4 Flash. That’s a 50x discount. Z.ai charges $0.26 per million cached versus $1.40 for GLM-5.2. Kimi K3 charges $0.30 cached versus $3.00 fresh. The pricing is designed to reward you for structuring your prompts in a cache-friendly way.

But here’s what most people miss.

Small cache gains cut your bill in half

This took me too long to internalize.

Cached tokens are billed at a fraction of fresh input price, usually 10x to 20x cheaper. That means the tokens you move from the “not cached” bucket to the “cached” bucket each save that full multiplier. The tokens that were already cached don’t change. So the cost savings from a cache improvement scale with the discount ratio, not with the percentage points moved.

Think of it this way. If cached tokens cost 1/10 of fresh tokens, then every token you move from fresh to cached saves 90% of that token’s input cost. Moving 5% of your total tokens from fresh to cached doesn’t save 5% of your input bill. It saves 5% times 9 (the discount factor minus one), which works out to nearly half of your fresh-token spend. The exact math depends on your starting cache rate and the discount ratio, but the principle holds: small cache improvements produce outsized cost reductions because the per-token discount is so large.

A concrete example. Say you send 100,000 input tokens per request on DeepSeek V4 Flash ($0.14/M fresh, $0.0028/M cached, a 50x discount). At 90% cache hit, 10,000 tokens are billed fresh. At 95%, only 5,000 are. You cut your fresh-token count in half by moving 5% of your total tokens into cache. The cached tokens themselves cost almost nothing, so the savings come almost entirely from the fresh tokens you no longer pay full price for.

Over 10,000 requests per day, that 5% cache improvement saves roughly $7 daily on the cheapest model in the lineup. Scale up to GLM-5.2 at $1.40/M input with 200K token contexts, and the same 5% jump saves significantly more. The math compounds because the discount multiplier is so large.

The general rule: if your cache discount is N (cached tokens cost 1/N of fresh), then a cache improvement of X percentage points saves approximately X times (N-1) divided by your current fresh-token fraction. With a 10x discount, going from 90% to 95% cache cuts your fresh-token spend in half. With a 20x discount, the same 5% improvement saves even more. The bigger the discount, the more painful every uncached token becomes, and the more a small cache improvement is worth.

This is why I stopped obsessing over per-token pricing and started obsessing over cache hit rate. The provider with the best cache infrastructure and the highest cache hit rate wins, even if their sticker price is higher.

Getting visibility with Bifrost

I needed logs. Not just “how many tokens did I use” logs. I needed to see, per request, how many tokens were cached, how many were fresh, what the cache hit rate was, which provider handled the call, and what it actually cost. Some providers give you this in their dashboards. Many don’t.

If you’re using a subscription like Z.ai’s Coding Plan or OpenCode Go (which serves open-source Chinese models), you get zero logs. No token breakdowns. No cache statistics. No per-request cost. You’re flying blind. The subscription feels cheap until you realize you can’t optimize what you can’t see.

Bifrost fixes this. It’s an AI gateway that sits between your agents and your LLM providers. Every request flows through it. It logs everything: provider, model, token counts, cached tokens, latency, cost, and the full attempt trail if it fell back between providers. The logging plugin runs asynchronously with less than 0.1ms overhead, so it doesn’t slow down your requests.

But logging is just the start. Once all your traffic flows through one gateway, you get a bunch of other things for free.

Installing Bifrost with Docker

The official Docker image is maximhq/bifrost. A docker-compose setup with persistent storage for logs and configuration:

services:
  bifrost:
    image: maximhq/bifrost:latest
    container_name: bifrost
    ports:
      - "8080:8080"
    volumes:
      - ./data:/app/data
    environment:
      - APP_PORT=8080
      - APP_HOST=0.0.0.0
      - LOG_LEVEL=info
    restart: unless-stopped

The ./data:/app/data volume mount is important. Bifrost stores its config database (config.db), request logs (logs.db), and optional vector store in that directory. Without the mount, all your logs and configuration disappear on container restart.

Start it up:

docker compose up -d

Open http://localhost:8080 in your browser. Everything Bifrost does is manageable from the web UI. Adding provider API keys, configuring routing rules and fallback chains, setting up virtual keys for governance, viewing live request logs with token counts and cache stats, monitoring costs per provider per model. You never need to touch a config file if you don’t want to. The UI applies changes in real time, no restart required.

If you prefer declarative setup or want to version-control your configuration, Bifrost also supports a config.json file that seeds the database on startup. But for most people, the web UI is the whole interface.

The logs database is SQLite by default, stored at /app/data/logs.db inside the container. If you want to query it directly, copy it out:

docker cp bifrost:/app/data/logs.db /tmp/logs.db
sqlite3 /tmp/logs.db "SELECT provider, model, prompt_tokens, cached_read_tokens, cost FROM logs ORDER BY timestamp DESC LIMIT 20;"

The logs table includes prompt_tokens, completion_tokens, cached_read_tokens, latency, cost, provider, model, status, and the full attempt_trail showing which providers were tried and why they failed. This is the data you need to actually optimize cache hit rates.

For production, Bifrost also supports PostgreSQL for the log store if you want to handle higher volume or run analytics queries. But SQLite is fine for most homelab and small-team setups.

Once Bifrost is running, point your agents at it instead of the provider directly. Bifrost speaks the OpenAI-compatible API, so you just change the base URL:

# Before
base_url = "https://api.deepseek.com/v1"

# After
base_url = "http://your-bifrost-host:8080/v1"

Now every request from every agent flows through Bifrost. You get logs for your direct API calls, your OpenRouter calls, your Z.ai Coding Plan calls, your OpenCode Go calls. Everything. Even the subscriptions that gave you zero visibility before.

Routing rules and fallback chains

Once your traffic is flowing through Bifrost, you can start shaping it. Configure which provider handles which model. Define what happens when a provider fails or hits a rate limit. Bifrost falls through to the next provider in your chain automatically. DeepSeek API having a bad day? Requests route to Z.ai or OpenRouter. Your agents keep working. You sleep.

No manual key juggling in your application code. You define the order in Bifrost’s web UI, and Bifrost follows it.

Stacking subscription accounts for more quota

One trick saves serious money if you’re using subscription-based providers. Some providers sell monthly subscriptions with per-account rate limits. OpenCode Go, for instance, charges $10/month per account. Each account gets $60 worth of GLM-5.2 usage but only $15 worth of Kimi K3 credits. Rate limits are $12 per 5 hours and $30 per week per account. One account caps out fast if you’re running multiple agents.

To stack accounts, you register each one as a separate provider in Bifrost (opencode-go-1, opencode-go-2, opencode-go-3, opencode-go-4) and configure a fallback chain across them. When provider 1 hits its rate limit, Bifrost’s routing rules fall through to provider 2. Provider 2 exhausts, provider 3 takes over. Your agents never see the rate limit. They just see requests succeeding.

Four OpenCode Go accounts cost $40/month total. For GLM-5.2, that’s $240 in effective usage credits. For Kimi K3, $60. The effective rate limit becomes $48 per 5 hours and $120 per week. That’s 4x the throughput of a single account for 4x the price, which sounds linear until you realize the alternative is hitting a wall and having your agents fail mid-task. The fallback chain turns four small pipes into one big pipe without any application-level changes.

This pattern works for any provider that allows multiple accounts. Z.ai’s Coding Plan, Kimi’s subscription, MiMo’s token plans. Register each account as a separate provider in Bifrost, set up routing rules to chain them, and Bifrost handles the fallback. The logging plugin records which provider served each request via the attempt_trail field, so you can see exactly when fallbacks happen and which accounts are doing the heavy lifting.

The math gets interesting when you compare stacking against direct API pricing. Four OpenCode Go accounts at $40/month give you $240 in GLM-5.2 credits. If your cache hit rate is 93%, those credits stretch further than the raw dollar amount suggests because most of your input tokens are billed at the cached rate. $60 in credits with 93% cache hit buys a lot more inference than the same $60 with 50% cache hit.

One caveat worth stating plainly. OpenCode Go can offer those huge discounts because they bulk-purchase API capacity at negotiated rates and reserve a slice of the provider’s GPU resources. But they only get a subset of the provider’s backends, not the whole fleet. During peak hours when their allocated backends are slammed by other users on the same shared pool, your cache takes the hit. Requests land on cold backends more often, more tokens get billed as fresh input, and your credits burn faster than they would on a direct API connection or a subscription with dedicated infrastructure like Z.ai’s Coding Plan. The discount is real, but inferior cache inflates the allowance you consume. If your workload is cache-sensitive and runs during Chinese peak hours, the cheaper credits can end up costing more per useful inference than going direct.

Cron-based peak and off-peak routing

Since Bifrost’s configuration is API-driven, you can automate provider switching on a schedule. A simple cron script can swap your primary provider to match peak and off-peak hours. During Chinese peak hours when DeepSeek charges double and Z.ai burns quota faster, route to a cheaper off-peak provider. When peak ends, switch back. Your agents don’t need to know anything changed. Bifrost handles the routing, the cron handles the timing, and you get the best price at every hour of the day without touching agent configs.

Virtual keys and budget governance

Bifrost lets you create virtual keys with their own budgets and rate limits. Give agent A a $50/month cap and agent B a $200/month cap. If agent A blows through its budget, it stops getting responses. You find out before the bill arrives, not after.

Single configuration point

Instead of managing API keys and base URLs across five different agent config files, everything lives in Bifrost. Add a new provider, swap a model, change a fallback order. One place. Every agent picks it up immediately.

Cost tracking per request

Bifrost calculates cost for every request based on the provider’s pricing and the token usage, including cached tokens at their discounted rate. You don’t have to build a cost tracker. You don’t have to guess what your bill will be. The data is right there in the logs database, per request, per agent, per provider.

The OpenRouter cache fragmentation fix

OpenRouter is a routing layer. When you send a request through OpenRouter, it load-balances across multiple backend provider instances. Each backend has its own KV cache. If your first request goes to backend A and your second request goes to backend B, backend B has no cache for your prefix. You get a cold start. You pay full price for tokens that backend A already processed.

OpenRouter has a fix for this: session-based sticky routing. If you pass a session_id in your request, OpenRouter uses it as a sticky routing key. All requests with the same session_id go to the same backend. The cache stays warm. The cache hit rate climbs.

Without a session_id, OpenRouter tries to identify conversations by hashing the first system message and first user message. This works for simple chat apps. It breaks for agentic workflows where the system prompt is stable but the conversation context changes between turns, or where multiple agents share the same provider but have different conversation histories.

The result: cache hit rates of 83-89% through OpenRouter versus 93%+ when going direct to the provider. That 4-10% gap is real money at scale.

The fix

We wrote a small Bifrost custom plugin (a Go shared object loaded via Bifrost’s plugin system) that injects a fixed session_id into every request body before it’s forwarded to OpenRouter. The plugin uses Bifrost’s HTTPTransportPreHook to modify the request body in-flight. One value, “bifrost”, for all traffic. Not per-agent, not per-session. One shared session ID.

Why one shared value instead of per-agent IDs? Because per-agent IDs fragment the cache pool again. Each agent gets pinned to a different backend, each warming its own isolated cache. A single fixed value collapses all traffic to one shared cache pool, matching the cache hit rate of going direct to the provider.

Bifrost’s plugin architecture supports this via Go plugins (.so files) loaded at startup. The plugin intercepts outgoing requests, adds the session_id field to the JSON body, and passes it through. OpenRouter reads the session_id, pins the request to the same backend, and the cache stays warm across turns.

You don’t need to write a plugin to benefit from this. If you’re calling OpenRouter directly, just add "session_id": "your-fixed-value" to your request body or set the x-session-id HTTP header. The key insight is: use one fixed value for all traffic from the same application, not per-conversation IDs.

After deploying this fix, our OpenRouter cache hit rate climbed from ~88% to match the ~93% we see on direct provider calls. Same models, same prompts, same traffic. Just one field in the request body.

Pinning the cheapest provider with presets

The session_id fix keeps your cache warm by pinning traffic to one backend. But OpenRouter still picks which backend that is, and it might not be the cheapest one. OpenRouter supports multiple backend providers for the same model, and it load-balances across them based on availability and price. If a cheaper backend drops out, OpenRouter routes to a more expensive one without telling you.

The sure-shot way to guarantee your requests land on the cheapest provider is to pin it explicitly. OpenRouter lets you specify a provider order or use a preset in your request. Set provider.order to a specific provider ID, or use a named preset that locks the routing. When you pin a provider, every request goes to that backend. No surprises on the bill.

If you’re routing through Bifrost, you can bake the preset into your provider configuration so your agents don’t need to know about it. Bifrost sends the request to OpenRouter with the provider pin already set. Your agents just see the OpenAI-compatible API. Bifrost handles the routing, OpenRouter handles the backend, and you get the cheapest available provider on every request.

Pinning also stabilizes cache a bit. When all your traffic goes to the same backend, the KV cache stays warm for your prefix. It’s not as reliable as a session_id for multi-turn conversations, but it eliminates the random backend selection that OpenRouter does by default. Combine a pinned provider with a fixed session_id and you get the closest thing to direct API cache quality while keeping OpenRouter’s fallback benefits.

When to call: Chinese provider peak hours

Here’s something most English-language guides skip. Chinese LLM providers run on Beijing time (UTC+8). Their peak usage windows are Chinese business hours. When their domestic users are hammering the API, cache eviction is faster, latency is higher, and some providers charge more.

If you’re in the US or Europe, Chinese peak hours often align with your evening or early morning. That means your off-peak is their off-peak, and you can ride the quiet hours for better cache retention and lower prices.

DeepSeek: peak surcharge

DeepSeek introduced peak-hour surge pricing for V4 models. Two peak windows in Beijing time:

• 9:00 AM to 12:00 PM Beijing (1:00 AM to 4:00 AM UTC)
• 2:00 PM to 6:00 PM Beijing (6:00 AM to 10:00 AM UTC)

During peak, input and output prices double. Off-peak, prices hold at standard rates. The cache hit discount (1/10 of standard input) still applies during peak, but the base price is higher, so cached tokens cost more too.

DeepSeek’s automatic prefix caching requires no opt-in. It just works. But during peak hours, higher traffic means cache eviction happens faster. Your carefully warmed prefix might get evicted before your next request arrives. Calling during off-peak hours gives you longer cache persistence on top of lower prices.

Z.ai (GLM): peak quota multiplier

Z.ai doesn’t publish explicit peak-hour surcharges on their per-token API. But their Coding Plan subscription applies a quota multiplier for advanced models (GLM-5.2, GLM-5-Turbo) during peak Beijing hours. A prompt that costs 1x quota off-peak might cost 2-3x quota during peak.

If you’re on the Coding Plan, this means your subscription quota burns faster during Chinese business hours. Save your heavy agentic workloads for off-peak and you effectively get 2-3x more mileage from the same subscription.

Z.ai’s per-token API pricing: GLM-5.2 at $1.40/M input, $4.40/M output, $0.26/M cached input. Cached input storage is currently free (limited-time promotion). Their cache infrastructure is solid. Going direct to Z.ai’s API gives 93%+ cache hit rates in our testing, compared to 83-89% through OpenRouter.

MiMo (Xiaomi): explicit off-peak discount

MiMo is the most transparent about off-peak pricing. They offer a 0.8x token consumption rate during off-peak hours:

• Off-peak: 12:00 AM to 8:00 AM Beijing time (4:00 PM to 12:00 AM UTC, previous day)

That’s a 20% discount on all token consumption during off-peak hours. If you’re in the US, this window is your afternoon to evening. Convenient.

MiMo V2.5 Pro and V2.5 are the current models. Pricing is token-plan based with credits. The off-peak discount applies automatically. No code changes needed.

Kimi (Moonshot): capacity, not pricing

Kimi K3 is Moonshot’s flagship at $3.00/M input, $15.00/M output, $0.30/M cached input. Moonshot doesn’t publish peak-hour surcharges. What they do have is capacity constraints.

In July 2026, Moonshot paused new K3 subscriptions after demand pushed GPU capacity to its limit. Existing subscribers kept access, but new signups were waitlisted. During peak Beijing hours, you may see higher latency and slower response times even if the per-token price doesn’t change.

Kimi K3’s cache hit pricing is aggressive: $0.30/M cached versus $3.00/M fresh. A 10x discount. If you can keep your cache warm, Kimi K3’s effective input cost drops dramatically. But cache warmth depends on the backend not being overloaded, which is harder during peak hours.

Moonshot also offers a subscription (Kimi Allegro) at $99/month, but it uses prompt-based quotas rather than per-token billing. The subscription doesn’t give you cache statistics, so running it through Bifrost is how you get visibility.

Quick reference: when to call

For US-based users, Beijing off-peak hours (roughly 8pm to 8am Beijing, which is 12pm to 12am UTC) align well with afternoon through late evening in American time zones. If you can batch heavy agentic workloads into those windows, you get better cache retention, lower prices, and faster responses.

What actually moves the cache hit rate

Beyond timing, a few things make a real difference:

Keep your prompt prefix stable. The system prompt, tool definitions, and any context documents should come first and stay identical between requests. The only thing that changes should be the latest user message at the end. If you reorder your messages, change your tool definitions between turns, or inject dynamic content into the system prompt, you break the prefix match and kill the cache.

Don’t interleave concurrent sessions on the same provider key. If two agents are hitting the same provider key simultaneously, their requests compete for cache space. On providers with limited cache memory (especially during peak hours), one agent’s prefix can evict the other’s. Use separate API keys per agent if your provider allows it, or stagger heavy workloads.

Avoid unnecessary context rotation. If your agent framework re-sends the full conversation history on every turn, that’s actually good for caching. The prefix grows but the early tokens stay cached. But if your framework truncates or summarizes older messages, the prefix changes and the cache breaks. Prefer frameworks that send the full history and let the provider’s cache handle the efficiency.

Route through a gateway that logs cache stats. This is the whole point of Bifrost. You can’t optimize what you can’t measure. Once you can see per-request cache hit rates, you can identify which agents, which prompts, and which workflows have poor cache performance and fix them.

The takeaway

The single most effective thing I did for my LLM costs was not switching providers, not changing models, and not negotiating pricing. It was getting visibility into cache hit rates and then optimizing for them.

Install Bifrost. Point your agents at it. Look at the cached_read_tokens column in your logs. If it’s low, fix your prompt structure. If you’re going through OpenRouter, inject a session_id. If you’re using Chinese providers, learn their peak hours and shift heavy workloads to off-peak windows.

A small cache improvement is a large cost cut. That’s not a metaphor. That’s the math.