AI / DRAFT OUTLINE
Evaluating AI beyond the demo
Questions for a future research note about reliable evaluation and useful failure analysis.
Research direction
This planned note will ask how an AI prototype should be evaluated before it becomes part of a workflow. The research and examples have not yet been completed.
Questions to investigate
Which tasks belong in the evaluation set? What constitutes an unacceptable error? How should human review and operating costs enter the comparison? References and findings will be added after research.
Planned reading & sources
Sources will be added after research and editorial review.