Free resource

Production readiness checklist for AI features

What must exist before a prototype meets real users — the questions worth answering before, not after, launch.

What happens when the model provider is slow or down — is there a fallback, or does the feature just hang?

Is there a timeout on every external call, with a defined behaviour when it fires?

Has the feature been tested under more than one concurrent user?

Is there a modelled cost per user, based on real provider pricing, not a guess?

Is there a hard cap or alert if usage spikes beyond the model?

Does the team know which line items in the model actually scale with usage?

Is there a scored evaluation set to check a prompt or model change against before shipping it?

Is there a way to tell when output quality regresses, other than a user complaint?

Has the failure mode — a wrong or unhelpful answer — been designed for, not just the happy path?

Is latency measured where the user actually feels it, not just server-side processing time?

Is there a dashboard, or does finding out about a problem depend on someone checking logs?

Is there a rate limit per user or session on any inference endpoint exposed to the public?

Has the feature been tested with adversarial or off-topic input, not just expected input?

If the feature supports more than one language, has each been tested separately, not assumed to work because English did?

Is the feature usable without a pointer, and does it degrade gracefully without JavaScript or WebGL where relevant?