EVALUATION SCOPE
Define what verified means.
An evaluation compares explicit conditions with authorized evidence. It does not measure every dimension of an agent’s quality.
01. Define what should happen
A task needs observable conditions: a file with specific content, an updated value, or an expected state. Vague goals cannot become precise verdicts without defined checks.
04. Record a scoped verdict
Each criterion has a result and evidence references. The outcome describes the conditions checked; it is not a guarantee about everything the agent did.
- Verified: evidence supports the condition.
- Failed: evidence shows the condition was not met.
- Insufficient evidence: a reliable decision cannot be made.
- Human review: ambiguity or conflicting evidence needs a person.
- Not applicable: the condition does not apply to this attempt.
Supported verification inputs
File, browser, desktop, HTTP and permission-state readbacks are represented by supported verifiers. Each task defines its expected conditions and authorized sources. Support is scoped to the configured verifier and the synthetic release environment.
Local verification evidence
The release readiness report records local Windows and Linux checks, PostgreSQL integration and reproducible evaluation against a frozen 100-case synthetic corpus. This corpus is a regression fixture, not a real-world accuracy claim.