Start with the current workflow
Document how work is done without the proposed AI step. Record elapsed time separately from active staff effort. Waiting for approval and editing an answer are different costs. Describe who checks the result, which errors matter and how an exception is resolved. If the baseline is unreliable, the comparison will be too.
Use a sample that represents the task you intend to change. Include ordinary cases and known difficult cases, but do not imply a small convenience sample represents every user or operating condition. Keep a record of selection criteria and exclusions. Protect personal and confidential information when preparing examples.
Define an acceptance rubric
A rubric makes “good” observable. For a hypothetical internal policy-summary pilot, reviewers might check whether the summary identifies the relevant policy, accurately states its conditions, cites the underlying section and declines to answer when the source is missing. These are example criteria, not universal requirements.
On a small screen, scroll the table horizontally to see all columns.
| Dimension | What to record | Why it matters |
|---|---|---|
| Task quality | Accepted, corrected or rejected against the rubric | Separates useful output from plausible text |
| Total effort | Preparation + generation + review + correction | Avoids counting generation speed as net savings |
| Exceptions | Reason, handling time and resolution | Exposes work displaced to other teams |
| Control failures | Unauthorized data, action or missing approval | Surfaces risks a quality average can conceal |
Keep the comparison fair
Where practical, evaluate the same task categories in both workflows. Record model and configuration changes so improvements are not attributed to the wrong intervention. A reviewer who knows which output is AI-generated may judge it differently; blind review can reduce that influence when feasible. Calibrate reviewers on a few examples before using their judgments as evidence.
Report the denominator. “Eight accepted results out of ten evaluated” is more useful than “high accuracy,” but still does not establish performance outside those cases. Show severe failures individually rather than hiding them inside an average. A single unauthorized disclosure can matter even when most outputs are useful.
Do not turn capacity into cash prematurely
Minutes saved can create available capacity, not automatically cash savings. Review workload, vendor charges, implementation effort and ongoing monitoring may offset part of the gain. Record observed effort changes separately from projected economics. Any financial model should name assumptions about adoption, task volume and how capacity will be used.
The end-of-week evidence packet
Keep the rubric, sample description, evaluation results, configuration notes and unresolved failures together. Record what was not tested. The next decision should be based on this packet, not a slide with the most impressive example.
Put your measures in the brief →