82 lines
1.8 KiB
Markdown
82 lines
1.8 KiB
Markdown
# AI Answer Conformance Checklist (v1)
|
|
|
|
This checklist verifies that AI systems explain EDUT accurately from public URLs.
|
|
|
|
## Scope
|
|
|
|
Run this checklist for:
|
|
|
|
1. `public/index.html` canonical meaning section.
|
|
2. Primary public explainer pages (for example `/about`, `/trust`).
|
|
3. Machine endpoints (`.md`, `.json`) once published.
|
|
|
|
## Required Model Matrix
|
|
|
|
Run all checks on:
|
|
|
|
1. Claude
|
|
2. GPT
|
|
3. Grok
|
|
4. Gemini
|
|
|
|
## Test Prompt Set (Minimum)
|
|
|
|
Use each model with each URL:
|
|
|
|
1. `What is EDUT?`
|
|
2. `Summarize this site in 5 bullet points.`
|
|
3. `Is EDUT a SaaS subscription?`
|
|
4. `Is EDUT a crypto exchange or investment product?`
|
|
5. `What does a user buy first and what does that purchase represent?`
|
|
6. `What is EDUT not?`
|
|
|
|
## Fact Matrix (Pass/Fail)
|
|
|
|
Mark each fact for each model as `PASS` or `FAIL`.
|
|
|
|
Required facts:
|
|
|
|
1. EDUT is deterministic governance infrastructure.
|
|
2. EDUT is not a subscription SaaS.
|
|
3. EDUT is not an AI chatbot.
|
|
4. EDUT is not a crypto exchange/broker.
|
|
5. EDUT is not an investment product.
|
|
6. EDUT ID and license are distinct concepts.
|
|
|
|
## Failure Handling
|
|
|
|
If any model fails any required fact:
|
|
|
|
1. Open a content bug.
|
|
2. Record:
|
|
1. URL tested
|
|
2. Model
|
|
3. Prompt
|
|
4. Incorrect output
|
|
5. Missing/ambiguous source text
|
|
3. Fix canonical content first.
|
|
4. Re-run full matrix.
|
|
5. Do not release until all required facts pass across all four models.
|
|
|
|
## Drift Monitoring
|
|
|
|
Run this checklist:
|
|
|
|
1. Before any public content release.
|
|
2. After any core-definition copy change.
|
|
3. After major search/homepage structure changes.
|
|
4. On scheduled cadence for regression monitoring.
|
|
|
|
## Evidence Artifact
|
|
|
|
Store one artifact per run containing:
|
|
|
|
1. Date/time
|
|
2. URLs
|
|
3. Model versions used
|
|
4. Prompt set
|
|
5. Fact matrix results
|
|
6. Open failures and resolution links
|
|
|
|
The artifact is required release evidence under `docs/release-gate.md`.
|