AI CostBase

Chatbot cost

The simplest stack of the eight — one model, many turns. The cost driver isn't the token price; it's how much context you re-send on every turn.

The formula

llm = convos × turns × (tokIn/1M × P_in + tokOut/1M × P_out) × 30 total = llm + platform per_convo = total / (convos × 30)

Token prices as of . Context grows with conversation length — the per-turn input default assumes a mid-conversation turn, not turn one. Prompt caching (re-sending the same long system prompt) can cut 30–70% off this number; excluded here as the conservative case. Human escalation costs are the real tail risk: one human-handled conversation can cost more than 100 automated ones.

The meter that moves is context, not the reply

Chat is the simplest of the eight calculators and the easiest to get wrong, because the intuitive cost model — conversations × messages × price — hides the thing that actually grows. Every turn re-sends the history plus the system prompt, so the input side of turn six is a different animal from the input side of turn one.

With the defaults here the input side runs about 3.2× the output side, and nearly all of that input is text the user has already seen. Two teams with identical conversation counts and identical models can face materially different bills purely because one handles questions in two turns and the other in eight.

Why conversation length beats conversation volume

Volume scales linearly and predictably: twice the conversations, twice the bill. Length does not, because context accumulates. The per-turn input is the sum of everything said so far, so a conversation that runs twice as long costs more than twice as much.

This is also why "average conversation length" is a dangerous input. Costs are concentrated in the tail — a small share of long, messy conversations carries a disproportionate share of the bill, and an average conceals exactly the segment you need to fix. Track the distribution before you set this number.

Prompt caching: the largest lever most teams never switch on

When a long, unchanged prefix — typically the system prompt and any fixed instructions — is re-sent on every turn, providers that support prefix caching can reuse the work instead of reprocessing it. For a support assistant with a 1,500-token instruction block, that is the majority of the input bill, repeated on every single turn.

It is deliberately excluded from this calculator, because cache discounts and minimum cacheable lengths differ by provider and by model, and assuming a discount you cannot claim produces a worse estimate than assuming none. Treat the result as the no-caching upper bound, and treat caching as upside you verify against your own invoice.

Where teams get this wrong

The tail: why the average conversation is a lie

Conversation cost is not evenly spread. Most conversations run two or three turns and cost almost nothing; a small minority run long, loop, or drag in history until the input side becomes enormous. Because the cost of a turn grows with everything said before it, that minority carries a share of the bill far larger than its share of traffic.

Which means the number that matters is not the average length — it is the shape of the distribution. If one conversation in ten runs four times the median length, that tenth can account for a third or more of the monthly total while remaining completely invisible in an average. Find the longest conversations you are actually having and look at what makes them long: usually an unresolved intent, a loop, or a customer who should have been handed to a person on turn two.