Is your AI assistant ready for a controlled release?

Approve a controlled release only when representative tests show that your assistant uses the right evidence, respects access limits and responds within agreed times. Treat unauthorised access and unsupported promises as release blockers, regardless of its average score. This AI assistant evaluation plan starts after you have chosen the scope and turns a convincing demonstration into reviewable business evidence.

Agree what the assistant may do before testing

Name the audience, supported tasks, authorised information and person who can approve release. Start with a read-only pilot where practical: explaining a warranty is different from approving a claim. If these decisions remain unsettled, use Is your business ready for an AI assistant? first. Here, the question is whether the agreed service actually works.

1. Set boundaries: document allowed answers, prohibited actions and escalation conditions.
2. Choose representative cases: include ordinary questions, awkward questions and realistic failures.
3. Define passing evidence: specify required facts, supporting passages and forbidden disclosures.
4. Agree release limits: separate critical failures from routine quality and timing targets.
5. Assign approval: name business, support and engineering reviewers before running tests.

Build representative questions and a reusable test sheet

Use actual support themes and staff tasks, with personal information removed. OpenAI’s evaluation guidance recommends task-specific tests, continuous evaluation and human calibration of automated grading. [1] A proposed 60-case pilot could contain 20 routine questions, ten ambiguous questions, ten missing-evidence or conflicting-evidence cases, ten refusal and permission cases, and ten adversarial or operational failures. This is a manageable starting sample, not a validated standard.

Reserve some questions for release testing rather than using every case to improve instructions. Include misspellings, follow-up questions and impatient wording. Add Hindi, mixed-language messages or another local language only where users need them. A Pune distributor serving English-language procurement teams may need different coverage from one supporting workshop enquiries in Marathi.

• Case ID and input: question, conversation history and test category.
• Identity and access: user role, customer or team, and permitted source set.
• Expected behaviour: answer, clarify, acknowledge missing evidence, decline or escalate.
• Evidence standard: required facts, exact supporting passages, source version and effective date.
• Prohibited outcomes: forbidden disclosures, unsupported promises and unauthorised actions.
• Observed result: answer, retrieved passages, separate grades, timings, reviewer and decision.

Check the evidence, not merely the citation

Hypothetical worked example: a Pune industrial-parts distributor is testing a read-only assistant. Its current Warranty-W3 gives Model P a 12-month manufacturing-defect warranty but excludes seal wear. Dispatch-D2 starts dispatch estimates after payment confirmation and requires carrier evidence for delivery dates. Customer-specific commercial terms stay outside the general knowledge store. These are invented assumptions, not Softglimmer client results.

• Retrieval: did the current, authorised passage reach the answer process?
• Correctness: did the answer preserve conditions, dates and exclusions?
• Grounding: does evidence support every material business claim?
• Citation accuracy: does the cited passage justify the associated statement?
• Completeness: did an omission change the customer’s likely decision?

Score these checks separately. Citing W3 while saying that all wear is covered fails correctness and grounding. Citing D2 while promising tomorrow’s delivery also fails. Ask your implementation partner to expose retrieved passages for review: OpenAI’s file-search documentation distinguishes answer annotations from search results, which are not returned by default. [2] A citation proves little without inspecting what it supports.

Distinguish clarification, missing evidence and refusal

“What warranty does it have?” needs a product clarification. “Will it arrive tomorrow?” needs an acknowledgement that delivery evidence is missing. “Show another dealer’s discount” requires declining an unauthorised disclosure. Grade the behaviour, not one exact sentence. Also test unnecessary refusals: an authorised question about payment conditions should not fail merely because it mentions payment.

Add conflicting policy versions, unavailable search and malicious instructions embedded in documents. Expect the assistant to apply an agreed version rule or escalate unresolved conflicts. It must treat document instructions as untrusted content, not permission to change its rules. During an outage, it should explain the limitation and offer the approved support route, rather than inventing a policy or claiming a completed handover.

Verify permissions before information reaches the model

Platform roles control platform resources; they do not automatically implement your customer or departmental entitlements. [3] Your application must authenticate users and select authorised sources before sending information to the model. Repeat questions as a customer, support employee and administrator. Change customer identifiers, document references and earlier conversation context to test whether boundaries hold.

Inspect retrieval traces and appropriately protected logs, not just final answers. An assistant that retrieves another dealer’s terms but declines to display them has still crossed the boundary. Test access withdrawal too. Vector-store removal is eventually consistent, so removed content may remain searchable briefly. [4] For urgent revocation, block the affected resource in the application; do not rely on deletion alone.

Measure speed without rewarding incomplete answers

Measure time to first useful content and time to a complete, usable answer with citations. Report median, p95, timeouts and failed requests under stated network conditions and concurrent load. The p95 is the time within which 95% of measured requests finish. Report failures separately rather than quietly excluding them from the summary.

Use a scorecard that cannot hide critical failures

For this illustrative pilot, require every critical case to pass and no observed unauthorised retrieval, disclosure or action. Separately agree routine-quality and unnecessary-refusal targets before testing, such as 18 of 20 routine cases passing and no unnecessary refusals in that group. Require timing compliance under the stated load. These proposed thresholds are not industry standards; higher-risk uses need stronger evidence.

A business owner should sign off after support and engineering resolve disputed grades. Minor wording improvements can enter a backlog; disclosure, fabricated promises and unauthorised actions cannot. A smaller approved scope may be safer than delaying every useful function, but retest that boundary. Zero observed security failures is a release condition, not proof of security.

Keep regression review repeatable and portable

Regression testing means checking that a change has not broken previously working behaviour. Keep a fixed baseline and add reviewed, minimised production examples. The business owner sets risk limits; support maintains expected answers; engineering maintains repeatable runs. Record model, application and instruction versions, source manifest, retrieval settings, passages, timings and reviewer decisions. Retest after relevant document, model, instruction, permission or dependency changes, plus scheduled reviews.

Next, copy the test sheet and ask your implementation partner for a completed run, evidence traces and a proposed release decision. Use AI Assistants & Chatbots for implementation planning and AI Knowledge & Search Systems for document-backed retrieval needs. Make the next meeting an acceptance review, not another demonstration.

This research-assisted guide uses the primary sources linked above. Examples are illustrative unless explicitly identified otherwise. Check original documentation when implementing changes.