Lesson 002Field guide · AI Safety & Privacy
Checking AI Outputs
Simple habits for verifying answers before you rely on them.
You'll learn
- The seven kinds of checkable claims hiding in AI answers
- The original-source rule for verifying anything factual
- Why asking the same AI "are you sure?" proves nothing
- How to match checking depth to the stakes
- A claim-extraction prompt that makes verification fast
01Section
From rule to routine
If you've taken What AI Can and Can't Do, you know the core rule: confident and correct are unrelated, and anything you'll repeat needs verifying. This lesson is the next step — turning "verify it" from good advice into a routine you actually run.
Because here's what happens without a routine: you know you should check, you're busy, the answer looks right, and you skip it. The fix isn't more willpower. It's knowing exactly what to check, where to check it, and how much checking each situation deserves.
Checkpoint
02Section
The seven things that need checking
Most of an AI answer doesn't need verification — structure, phrasing, and general reasoning are yours to judge directly. Verification is for the checkable claims hiding inside. Seven types cover nearly all of them:
Facts
Any statement about the world that could be false. "The deadline is April 15" either is or isn't.
Numbers
Statistics, percentages, prices, quantities. AI produces plausible numbers, not confirmed ones.
Names
People, companies, products, places. Close-but-wrong names are a classic slip.
Dates
Events, deadlines, releases, history. An off-by-a-bit date reads exactly as confidently as a right one.
Quotes
Exact words attributed to real people. AI paraphrases and invents attribution fluently.
Links & citations
URLs, book titles, studies. Invented often enough to deserve a click every single time.
Calculations
Multi-step math can go quietly wrong in the middle. Recheck with a calculator, not by rereading.
Checkpoint
03Section
The original-source rule
Plain-language definition
Original source — a place where the fact lives independently of the AI — the official website, the actual document, the person involved, a calculator, the primary study. If the AI is wrong, the original source doesn't inherit the error.
The rule: verify checkable claims against an original source, never against more AI output. A deadline gets checked on the official site. A quote gets checked against the article or recording. A calculation gets redone in a calculator. A claim about your own data gets checked against your own files.
Verification that isn't
Asking a second AI tool the same question and getting the same answer feels like confirmation. It isn't — both tools learned from similar data and share similar blind spots. Two fluent guesses that agree are still guesses.
Links deserve extra care: an AI-provided link can be fabricated, or real but pointing at a page that doesn't say what the AI claims. Click through and read the actual page before you cite it.
Checkpoint
04Section
Why "are you sure?" isn't verification
The most tempting shortcut is asking the same AI to double-check itself. It feels rigorous. It isn't — for a simple reason: the same process that produced the error is now being asked to find it. The model doesn't have a separate fact-checking mode. It generates another likely-sounding response.
Ask "are you sure?" and you'll see one of two things. Sometimes it confidently repeats the mistake. Sometimes it apologizes and "corrects" itself — even when the original answer was right. Either way you've learned nothing about the truth. You've learned the model responds to social pressure.
What the AI can do for you
It can't verify its own claims, but it can list them. Asking it to extract every checkable claim turns a vague "check this somehow" into a concrete to-do list — which you then verify at original sources.
Copy and keep — the claim-extraction prompt
List every specific claim in your previous answer that I should verify before relying on it — every fact, number, name, date, quote, and link. For each one, tell me the kind of original source that could confirm it (official site, document, calculator, the person involved). Don't re-verify anything yourself — just give me the checklist.
Checkpoint
05Section
Match the depth to the stakes
Not everything deserves the full treatment — checking everything equally is how verification dies by Friday. Let the stakes set the depth:
Low stakes: skim: Brainstorms, your own notes, drafts nobody else sees. Read it, keep what's useful. Verification here would cost more than an error would.
Medium stakes: check the load-bearing claims: Anything leaving your desk with your name on it. Find the facts the message depends on — usually two or three — and verify those at original sources.
High stakes: verify everything, then add a human: Money, legal language, health, anything public or hard to reverse. Every checkable claim gets an original source, and a qualified person reviews before it ships.
Key takeaway
Verification isn't a tax on everything. It's focused effort on the claims that could actually cost you.
Pause and think: Think about the AI outputs you used this week. Which tier did each belong in — and did your checking match?
Checkpoint
Prompt exercise
Run a real verification pass
Copy this prompt into ChatGPT, Claude, Gemini, Copilot, or whichever AI tool you have access to — the website doesn't run AI itself. Ask about a topic you'll genuinely use, then verify the claims list it gives you at original sources. Expect a mix: most right, some shaky. That mix is the lesson.
Answer this question for me: [your question — pick something with real facts, numbers, or dates in it]. Then add a final section called CHECK THESE: list every specific fact, number, name, date, quote, and link in your answer, each with the kind of original source that could confirm it. Be honest about which claims you're least certain of.
Reflection: How many claims made the list — and how many survived checking? Remember that ratio next time you're tempted to skip the pass.
Quick check
4 quick questions — no pressure
There's no pass or fail here. Answer them all, and we'll show you the answers either way.