What to check before moving an AI pilot into production

List the tasks the system may perform, the people who will use it and the consequences of an incorrect output. Agree measurable quality, latency and cost expectations. Different tasks may need different acceptance criteria; a single overall accuracy figure can hide important failures.

• Include normal inputs, ambiguous requests, missing information and relevant edge cases.
• Define the expected answer or decision and how it will be assessed.
• Test permission boundaries and whether retrieved information is current and appropriate for the user.
• Record the model, configuration and source versions used in each evaluation.
• Keep a reviewable set of failures and repeat it after material changes.

Illustrative example: an assistant that drafts a support response may begin with mandatory agent review. Approval to draft does not also authorise refunds, account changes or outbound messages. Evaluate each additional action separately.

A readiness decision should record the tests completed, unresolved risks, access arrangements, receiving owner and conditions for expanding use. Treat a promising demonstration as an input to that decision, not a substitute for it.