6 Questions for Any Claim, from Anyone

The advice was right, the instructions were missing

An AI assistant asked whether it should be trusted answered that nobody should treat its default summaries as truth, and that the way to use it properly was to "demand the specific titles of the original studies, court filings, or data sheets."¹

Good advice. Almost nobody knows how to follow it, because "check the primary source" is a slogan rather than a procedure.

This is the procedure. It works on an AI answer, a broker, a comparison site, a product illustration, an article, and on this Academy.

1. What document says this?

Not "what is the source" — sources can be a person, a category, a vibe. What document. A title, a publisher, a date.

If the answer is "studies show", "experts recommend", "research suggests" or "it is widely understood", you have received a default answer. Those phrases are the sound of nobody having opened anything.

The follow-up that does the work: can you give me the title and the date?

2. Who made the number, and do they sell the thing?

Every figure was produced by someone with a reason to produce it.

This is not a rule to distrust everyone. It is a rule to know which you are holding:

• A regulator or agency measuring an industry it oversees — usually the strongest number

available.

• Peer-reviewed research — strong, and check the funding.
• A trade body or company measuring its own market — sometimes the only figure that exists,

and use it while saying whose it is.

• A comparison site paid per referral — a marketing surface with a table on it.

And apply it evenly. If you discount a consumer group for having a cause, discount an insurer's own report for having a market. The asymmetric version of this test is where most bias lives.

3. Is this the document, or somebody's summary of it?

Most wrong numbers in circulation are correct numbers that were retold.

A figure passes from a report to a press release to an article to a blog post to an answer, and somewhere in that chain a median becomes a mean, a six-month figure becomes an annual one, or a 2017 count becomes a present-tense fact.

Every one of those happened in material we checked ourselves.

The test is one question: can I get to the thing itself? Most government and regulator documents are free. Most academic papers have a free version. If a number cannot be traced to anything you can open, you do not have a fact — you have a rumor with a decimal point.

4. What is this number actually of?

The most common error is not a wrong figure. It is a right figure describing something else.

Watch for:

• What period? A six-month figure quoted as annual.
• What denominator? A share of premium, or of revenue, or of the amount financed?
• Net or gross? Before or after what?
• Per what? Per claim, per person, per household, per policy?

None of that requires expertise. It requires reading the column heading.

5. What would change my mind?

Ask it of the claim, and then ask it of yourself.

A claim that nothing could disprove is not a strong claim. If a product's supporters treat every bad outcome as misuse and every good one as proof, the claim has stopped being about the world.

And the version pointed inward is the more useful one. If no possible figure would change your view, you are not evaluating — you are defending. That is worth knowing before a large purchase.

6. What is missing?

The hardest and most valuable one.

An answer is shaped as much by what it leaves out. Common omissions in financial information:

• The cost of the thing being recommended.
• Who pays the person recommending it.
• The people it did not work for.
• The alternative that costs nothing.

A recommendation with no downside stated is incomplete, whatever else it is.

One thing that measurably works

There is a practical finding worth knowing, because it is unusually concrete.

When researchers tested how often AI systems recommended a sponsored option, simply asking for a neutral comparison table first — a single short instruction, before the real question — cut sponsored recommendations from 46.9% to 1.0% across ten open models.²

One sentence, added before you ask. It costs nothing and it is the single highest-return habit in this chapter.

Applying this to us

These questions work on this Academy and you should use them here.

Every figure in these chapters names a document with a date. Where the number is a trade body's own, we say so. Where a figure could not be confirmed, the chapter says unconfirmed rather than filling the gap. And Plenee is a company writing about the industry it competes in — question two applies to us as much as to anyone.

If a chapter here fails these tests, it is wrong and worth telling us about.

The short version

Six questions. What document. Who made it and do they sell the thing. Is it the source or a retelling. What is the number actually of. What would change my mind. What is missing. They work on software, salespeople, articles and us, they need no expertise, and the highest-return habit of the lot is asking for a neutral comparison before you ask your real question.