From PoC to production: why most AI projects never ship

The demo took three weeks and looked great. Eight months have passed since and it is still not in production. If that sounds familiar, you are not alone — and it is not your fault. The road from prototype to operation is a different discipline from building a prototype, and it is usually forgotten at budgeting time.

Here is what that space actually requires.

Why a demo is misleading

A prototype is built on data someone selected. Usually, without realising it, they pick the examples that make sense — legible, complete, typical. In production you get an email with no subject, an invoice photographed at an angle and an order with a note that contradicts the rest of the message.

A prototype is also tested by one person who knows what to expect and unconsciously phrases things so that it works. A hundred users phrase things differently.

And third: a prototype has no running cost attached. While only you use it, the cost is negligible. At a thousand requests a day it is a different conversation.

Step 1: evaluations, before anything else

This is the most important sentence in the article: until you can measure quality, optimising anything is pointless.

An evaluation is a set of inputs with expected outputs that runs automatically on every change. It sounds like testing in ordinary software, because it is — with the difference that the output is not deterministic, so you score accuracy rather than equality.

Without it, development looks like this: you change a prompt, two things improve, one quietly breaks, and you find out three weeks later from a customer. With it: you change a prompt, you see a number, you decide.

The practical minimum is 30–50 real cases with the correct outcome. Not invented ones — real ones, including the ugly ones. The ugly ones are precisely why evaluations exist.

Step 2: what happens when it fails

A prototype has no failure states. A production system needs one for each of these:

• The model provider has an outage. Do you have a fallback? Or at least a sensible error message instead of an endless spinner?
• The response comes back in the wrong format. Do you retry? How many times? And then what?
• The model is unsure. Is there a queue for a human, or do we pretend it was sure?
• A request takes too long. Is there a timeout, and does the user know about it?

In a prototype most of these are handled by not handling them. In production they are exactly what determines user trust.

Step 3: costs, before they surprise you

The token bill has an unpleasant property: it grows with usage, which means with success. The better it works, the more it costs.

What we do about it in practice:

• Caching. Repeated parts of the context (system prompt, documentation) do not need to be sent in full every time.
• Shorter context. Sending twenty retrieved documents when five suffice is needlessly expensive and less accurate.
• Routing to a smaller model. Simple requests do not need the largest model. Classifying complexity up front often pays back well.
• Limits and alerts. A daily cap and a notification when it is exceeded. This is five minutes of work and occasionally saves an unpleasant surprise.

Step 4: versioning, so you can roll back

A prompt is code. A model is a dependency. Both change and both must be versioned.

Concretely, for every production response you should be able to say which prompt version and which model produced it. Without that, a report saying “it worked better last week” can neither be verified nor fixed.

Also: when a provider ships a new model version, do not assume it will be better in your case. Re-evaluate. We have seen cases where a newer version was stronger overall but worse on a specific narrow task.

Step 5: who is going to operate it

The quietest cause of failure. The solution is deployed, the vendor leaves, and nobody inside the company can change a prompt or read the logs. Six months later the system is switched off because “it does not work” — with nobody having found out why.

That is why handover includes a runbook: what to do during an outage, where the logs are, how a change is deployed, who to call. Plus training for the person responsible.

How to tell you are production-ready

The checklist we use:

• We can measure quality on a set of real cases.
• We know what happens for each type of failure.
• We know what one request costs and a cap is in place.
• We know which prompt and model version is deployed.
• We monitor latency, error rate and cost.
• There is a person inside the company who owns the system.

If you can tick all six, you are further along than most.

Summary

The road from prototype to production is not “just deploy it”. It is a distinct phase, often comparable in length to the development itself — and skipping it is behind most AI projects that get quietly switched off after a year.

If you have a prototype you are afraid to release, AI deployment and operations is exactly the missing part. We also take over solutions built by someone else.