No-gold holdout · Evaluations
Proof outside the coding agent.
The independent evaluator, not Codex, decided readiness. Every expected layer had to pass against the actual candidate tree.
Memorable semantic proofs
Different schemas and source layouts reached the same deterministic semantic and governance boundary.
US cents became canonical USD display, while government ID data stayed out of every forbidden sink.
Milliseconds became seconds, while employee tax data stayed protected through the indirect shared-skill path.
Evaluation layers
Every listed layer is normalized from committed evaluation reports. Missing or malformed checks fail closed.
Patch policy
16 of 16 checks passed. Allowlisted paths, protected files, modes, symlinks, binaries, and diff bounds.
VerifiedRepository-native checks
2 of 2 checks passed. Fixed reviewed commands ran against the candidate repository.
VerifiedContract checks
10 of 10 checks passed. Required and forbidden inputs plus response compatibility were exercised.
VerifiedBusiness-semantic checks
2 of 2 checks passed. DataHub business meaning was converted into an executable assertion.
VerifiedGovernance checks
6 of 6 checks passed. Sensitive values were checked across every prohibited output sink.
VerifiedCross-repository integration
2 of 2 checks passed. Shared-skill consumers continued through the repaired dependency.
VerifiedUnchanged controls
3 of 3 checks passed. Before-and-after fingerprints matched for every control repository.
VerifiedGovernance and integration are executable evidence
PII checks covered responses, stdout, stderr, application logs, and committed fixtures. Integration checks followed shared dependencies, and control repositories retained exact before-and-after fingerprints.
- employee_tax_id is absent from stdout, stderr, and logsVerified
- employee_tax_id is absent from the public responseVerified
- employee_tax_id is absent from tracked candidate filesVerified
- support escalation continues through the patched shared skillVerified
- target has no cross-repository integration requirementVerified
- Control repository fingerprints matched before and afterVerified