If the AI feature you shipped has a bill that climbs faster than the customers using it, that is a running cost problem you did not plan for, and it gets more expensive to fix the longer it runs.
Most people who message me about a runaway AI bill assume the fix is a cheaper model. That is rarely the real lever. The bill is usually driven by how much text gets sent and returned on every single call, not by which model answers it. A wrong or vague answer is not free either. If a customer has to ask again, or someone on your team has to catch and fix a mistake, that cost is bigger than whatever you saved switching models.
The fix, when the architecture is the actual problem, is rarely a full rebuild. It is usually finding the two or three places where more text moves than needs to, and closing those gaps. That is the kind of work I do under my AI Development service: look at what is actually driving cost, and fix the architecture, not only swap the model and hope. Left alone, a bill that already outpaces usage keeps growing every month you add customers on top of the same inefficiency.
I have built AI features for a veterinary data platform processing 2 million-plus records and for my own product used by 30-plus people daily, and in both cases the cost work was the same: find where text piles up, then cut it without changing the answer the customer sees.
TL;DR
- The bill is driven by volume, not by the model's sticker price. Every word sent in and returned is billed. Long conversation history, long documents, and long answers add up fast.
- Five changes cut cost without hurting quality: reuse repeated context instead of resending it, match the model size to the task, batch work that is not urgent, cap how long answers are allowed to run, and trim what history actually needs to be sent.
- Switching to a cheaper model is the last lever, not the first. Test it before you ship it. A wrong answer a customer has to redo costs more than the tokens you saved.
- Cost control is a habit, not a one-time fix. The same discipline that kept a sync job under 50 seconds for one of Brazil's largest veterinary networks applies to a chatbot or an AI feature inside your product.
Table of contents
- What actually drives the bill
- When switching to a cheaper model backfires
- What cost discipline looks like in a real build
- Building the habit before it becomes a problem
- For your engineer
- FAQ
What actually drives the bill
The number that moves your invoice is how much text passes through the model, in both directions, on every request. Not the brand of the model. Not how clever the feature is. Volume.
Four things quietly inflate that volume:
Chat history resent on every turn. A conversational AI feature does not remember anything between messages by default. To keep the conversation coherent, the full history gets sent again with every new message. A ten-message conversation can mean the first message gets billed ten times over.
Long context used out of habit. If you paste an entire document, a full customer record, or an entire codebase into a request when only one paragraph of it is relevant, you pay for all of it, every time.
Retries on failure. When a request times out, returns something malformed, or gets rejected, most systems silently try again. Each retry is a second full bill for the same question.
Unbounded answers. If nothing caps how long the model is allowed to respond, some answers run far longer than needed. Verbose output is not free.
None of these four are model problems. They are architecture problems, and architecture problems have a fix that does not touch the model you picked.
When switching to a cheaper model backfires
The obvious-looking fix, drop to a cheaper model across the board, is the one I recommend last, and only after testing.
A wrong or vague answer is not free. If a customer has to ask again, or a mistake needs a human to catch and fix it, that cost is bigger than whatever you saved on the per-request rate. The right sequence is: measure how the current model performs on your actual requests, test the cheaper model against the same requests, and only switch where the answers hold up.
A cheaper model is a legitimate option. It is only the fourth or fifth thing to try, after volume is already under control, not the first.
What cost discipline looks like in a real build
I built the sync layer for one of Brazil's largest veterinary networks, moving 2 million-plus records across four systems into a single source of truth, with each sync completing in under 50 seconds. That project was not an AI feature, but the underlying discipline is identical: know exactly how much data moves through every step, and cut what does not need to move.
I apply the same thinking on Instill, a knowledge base product I built and run myself, used daily by 30-plus people and holding over 1,000 saved skills. A product like that lives or dies on cost per request staying predictable as usage grows, which is why the information sent to the model on each query is scoped to what that specific query needs, not the entire knowledge base.
I also built a custom CRM for Norte Web Digital that grew its lead base by 500 percent while keeping the lead-to-customer cycle down to 3 to 4 days. Systems that handle that kind of volume only stay affordable if every automated step, AI-driven or not, is scoped tightly from the start.
Building the habit before it becomes a problem
The cheapest time to fix a cost problem is before you ship. Once a feature is live and customers depend on it, every change carries more risk, and a habit of resending full context or skipping a cap on response length gets expensive fast once usage grows.
If you already have an AI feature in production and the bill is climbing faster than usage, this is the work I do under AI Development: I come in, look at what is actually driving your cost, and fix the architecture.
For your engineer
The rest of this section is written for the person who will actually implement these changes: your developer, or whoever you bring in for the project.
What a token actually is. A token is the small chunk of text an AI model reads and writes, and the unit every provider bills by. Rough rule of thumb: one token is about three-quarters of an English word, so 1,000 words lands around 1,300 tokens. Providers charge a rate per million tokens in and a separate, usually higher, rate per million tokens out. Both OpenAI's pricing page and Anthropic's pricing page publish live per-token rates.
Five levers that cut cost without cutting quality, in the order to reach for them.
- Reuse repeated context instead of resending it. If the same system instructions or background document get sent on every request, that repeated block can often be cached on the provider's side so it is not billed full price every time. Anthropic documents this in its prompt caching guide, and it is one of the highest-impact changes available, since it requires no change to the product experience at all.
- Match the model size to the task. Not every request needs the most capable model available. Classifying a support ticket or extracting a date from an email is a simple job. Save the larger, more expensive model for the requests that actually need reasoning.
- Batch what is not urgent. If an answer does not need to come back in the next 2 seconds, batching a group of requests and processing them off the live path is usually billed at a steep discount. OpenAI's batch API documentation is a clear example, and it is a good fit for nightly summaries or bulk tagging.
- Cap how long an answer is allowed to run. Setting a maximum output length stops the rare runaway response from turning into a rare runaway bill. This is a one-line setting in most implementations.
- Trim what history actually needs to be sent. A conversation does not need every prior message resent in full. Summarizing older turns, or dropping ones no longer relevant, keeps the bill proportional to what the conversation actually needs.
Every one of these five reduces volume. None of them requires accepting a worse answer.
FAQ
What is the single biggest driver of a high LLM bill?
Chat history and long context sent on every request. A conversation that resends its full history on each turn, or a prompt that includes a whole document when one paragraph would do, moves far more text than the answer itself requires. Fixing that one habit usually has the biggest effect on the invoice.
Does switching to a cheaper model always save money?
Not always, and not by itself. A cheaper model that gives a worse answer can cost you more once you count the rework, the follow-up question, or the customer who has to try again. Test the cheaper model against your actual requests first, and only switch where the quality holds.
What is prompt caching and does it help small products?
Prompt caching lets you avoid paying full price for the same block of text (like system instructions or a reference document) every time it repeats across requests. It helps any product where the same context gets reused often, including a small one, since the savings scale with how often that block repeats, not with the size of your business.
How do I know if my AI feature's cost is actually a problem?
Compare your cost per request against how much that request is worth to you. If a support answer costs pennies and saves a human 5 minutes, that is a good trade regardless of the sticker price. The problem cases are usually retries, oversized context, or unbounded answers driving cost up without adding value.
Can I control AI costs without an engineer on staff?
Yes, at least at first. Capping output length and picking the right model size for each task are configuration choices, not deep engineering work. Prompt caching and context trimming take more care to do well without breaking the experience, which is where bringing in outside help for a focused pass tends to pay for itself quickly.
Next steps
If your AI feature's bill does not match what it delivers, the fix is almost always architectural, not a different model. I look at where your text volume actually goes and cut what does not need to be there. Let's talk about your project, or see the full scope of what I build under AI Development and my other services.