‹ All projects
DesignLLM testing

Hallucination checker

Splits an answer into single claims and checks each one against the source.

The problem

“Did it make something up?” cannot be measured for a whole paragraph. One sentence can be right and the next invented.

How it works

  1. 01Split claims
  2. 02Find evidence
  3. 03Supported?
  4. 04How serious?
  5. 05Flag

What was hard

  • Deciding what counts as “supported” when the source only implies it.
  • A made-up policy clause is worse than a wrong date — severity matters.

The goal

Each unsupported claim is shown next to the evidence that should have backed it.

This is a design I worked out on paper — the problem, the approach and the trade-offs. It is not a shipped product.

Built with

Claim extractionEvidence matching