Shipping it. The unglamorous half.

Build it from real traffic, not imagination. Take thirty actual requests, including the ones that went badly, and write down what a good answer would have been. The failures are the valuable half — anybody can pass the easy cases.

Then run it on every prompt change, every model change, and every retrieval change. If a change improves twenty-eight and breaks two, you now get to make an informed decision instead of an optimistic one.

Exact match works for classification and extraction. For prose, the practical approach is to have a model grade the output against your reference answer with a short rubric — not perfect, but consistent, and consistency is what you need to detect change.

Whatever you do, hold the grader still. If both the thing being tested and the thing testing it move at once, the number means nothing.

The nasty version is indirect: the attacker never talks to your application at all. They put the text in a document, a web page, a support ticket, a code comment, a calendar invite — anywhere your retrieval might one day pick it up. Then they wait. Your own user, doing something entirely innocent, triggers it.

White text on white background, a zero-width character run, an HTML comment, alt text on an image, metadata in a PDF. A human reviewing the document sees nothing wrong. The parser that fed your pipeline saw the lot.

Strip markup and normalise whitespace before indexing, and be aware that this reduces the problem rather than removing it.

The distinction is the whole thing. A prompt defence reduces the probability of being fooled. A capability defence reduces what being fooled achieves. Only the second is something you can put in a risk assessment and defend, because only the second still holds when the model does something nobody predicted.

Design as though every model call may at some point be under the attacker's control, and ask what that gets them. If the answer is "reads a public document and writes a summary", you are fine. If it is "sends mail as the company" or "runs SQL on the production database", the fix is not a better prompt. It is a smaller tool.

A short checklist that catches most of it. Can this tool read anything outside the current user's scope? Can it write, send, delete or spend? Would you let an anonymous member of the public call it with arbitrary arguments — because that is what you have built if a retrieved document can steer it?

Scoped reads can run unattended. Anything with a side effect — money, mail, deletion, publishing — should either be irreversible-proof or need a person. And put the confirmation in your own UI showing the real parameters, not in a sentence the model composed. The model's summary of what it is about to do is not evidence of what it is about to do.

Caching usually wins outright, because most prompts have a large unchanging front. Model choice is next: classification and routing rarely need your best model, and a small one is often several times cheaper. Output length matters more than people expect, since output costs several times input.

Capping the loop is the one that prevents disasters rather than saving pennies. An agent that retries a failing tool will do it until something stops it, and nothing will stop it.

Retrieval that pastes in twenty chunks when five would do. A conversation with no truncation, growing linearly for as long as the tab is open. Thinking left on for a task that never needed it. A retry loop with no ceiling. Each is invisible until you attribute cost per feature rather than watching one total.

Then look at what happens before the model is even called. Retrieval, reranking, tool calls and permission checks all land in front of that first token, and they are often the bulk of it. Do the independent ones in parallel rather than in a row.

Averages hide the problem. Your p95 is where the long documents, the multi-step tool runs and the retries live, and it is what a meaningful share of users get. A feature with a good median and a dreadful p95 reads to users as unreliable rather than slow.

Prompt version is the one people leave out and then need. Without it you cannot tell whether last Tuesday's complaint came from the prompt you have now or the one you replaced. Treat prompts as versioned artefacts, not strings that happen to live in source.

Those logs contain whatever your users typed, which will include things they would not want retained. Redact obvious identifiers on the way in, set a retention period, and put the same access controls on them as on the source data. A debugging convenience that quietly becomes a second copy of your customer database is its own kind of incident.

Write down, per feature, what happens when the model is unavailable, when it returns nonsense, and when it refuses. "Show an error" is a legitimate answer if you have chosen it deliberately. The failure mode to avoid is the one nobody considered, which is usually a spinner that never resolves.

If the reason for failure is capacity rather than a total outage, dropping to a smaller model keeps the feature alive at reduced quality. Decide in advance whether reduced quality or an honest error is the better outcome for that particular feature — for a summary it is usually the former, for anything a decision rests on, usually the latter.

Redact what you can before it leaves — identifiers, payment details, health information — and check whether your provider offers a zero-retention arrangement if your obligations need one. If they genuinely cannot be met, that is the honest argument for running a model yourself, and about the only one that always holds.

Thirty real cases including the awkward ones, run automatically. Until that exists, nobody knows whether it works.

Restrict the tools, scope the credentials, require approval for anything with a side effect. Then the prompt defence is a useful extra layer rather than the only layer.

The identity comes from the session, never from the model. If a tool needs to know who is asking, tell it yourself.

Cap the iterations and the spend per request. Fail loudly at the cap rather than quietly continuing.

Sample real traffic and keep whole traces. Reading twenty of them answers most questions that a dashboard cannot.