Expensive gaping canyon fact on API costs vs self hosting

17 Aug, 2026

Money must be funny, but token economics are funnier

To the reader, let me spoil you

If you do significant agentic coding work, or run an enterprise and you want an AI native programming team, or you just like to read about token economics, then this party fact is for you. Just don't estimate your invoice bill based on the token counts by list price alone, since surprise, surprise, caching is like a golden payback program for the API pricing. ~end note

Like every proud mama duck that a product manager tries to subtly emulate with every small success, I'd like to showcase a quick price comparison discovered during our team's work on benchmarking the 64 SWE-BenchPro tasks on 4 different models.

When running the tasks set via Anthropic's API for Claude Opus 4.8, based on our director of engineering's credit card bill, the cost was estimated at $98 (we had two runs, one failed towards the end, so total cost was 200 something euros. Yes, we converted to $, don't raise your sovereign pitchforks yet). Recently we looked at the traces and calculated based on the Anthropic's price per input for $5 / MTok (input_tokens: 114.514.412) and output tokens for $25 / MTok (output_tokens: 1.159.974), we end up with a whopping amount of $600.75.

To confirm we are not having delusions when looking at this canyon in price difference, a short calculation¹ of the cache reads and writes based on the trajectories ended up confirming the price of the API was precisely $106.76 (not far from our eyeball estimations when deciphering the API cumulative cost billed. Sorry for not being terribly precise this time, we will buy a cup of coffee and a croissant for the $8.76 difference).

I ran similar estimations for Kimi K3. Unfortunately we haven't run on the Moonshot API since our director of engineering is on well deserved vacation and we don't have access to his credit card. But looking at the trajectories that gives us all the details on input/output tokens and cache read & write plus their pricing from the website we would end up somewhere between $80-120 for the API run. Knowing our self hosted run cost around $450, it is difficult to chew on the following conclusions without more questioning:

1. At our utilization, on this workload (so take it with a grain of salt) running via an API is about 80% cheaper than renting GPUs on neoclouds.
2. Self hosting without cache is a losing race against APIs, especially when it comes to agentic coding tasks (at this moment in time when this blog is written (Aug 2026) )
3. I doubt sovereignty is the sole argument that can be made when justifying why one should be self hosting. I usually run the cost analysis for fun for my team, since we discover daily insights, and some of them don't end up getting posted yet. If you want to read deep dives, go check us out at aistack.imec-int.com! Ruthless self promotion, but I am god damn proud of my team's work (and mine, of course).

Small ending note

To be clear, I do not advocate for only API usage or only self hosting, that is a whole other debate. But I like discovering facts. And while this is a well known fact in the AI industry, that the API price doesn't match the amount of tokens consumed (the raw token math came out ~5x higher than our actual bill, thanks to prompt caching), is always worth backing up with real experiments until it becomes mainstream.

Footnotes

#AI #experiment notes #tokens