How false does this read?

Downloading the model (about 125 MB, cached after the first visit)…

No reading yet.

threshold

reads like a verified claim reads like a debunked one

Call it false above 0.50 0.50 is just the convention. On the test split 0.25 gives the best F1, so lowering it catches more and raising it makes the model more selective.

Try one of these — the first four are real LIAR statements

Chunk by chunk

Long text is split into passages and scored separately, because the model only reads 128 tokens at a time. Most suspicious first.

What this actually measures

Not truth. The model has no retrieval, no knowledge base, and no access to anything that happened after about 2016. It learned which phrasings tend to accompany a "false" verdict from PolitiFact, and that is all it reports.

On the 1,283 held-out statements:

Accuracy	0.645
Macro F1	0.635
Precision on FAKE	0.775
Precision on REAL	0.503

Answering "false" every single time would already score 0.641 accuracy on that split, which is why macro F1 is the figure worth reading and accuracy on its own is nearly meaningless here. Charts, the per-class breakdown and the error analysis are on the model page.

Where it goes wrong

Its most confident mistakes are true statements about contested political subjects. It appears to read charged political language as a sign of falsehood. That is a real bias in the model, visible in the data, and worth knowing before you trust any single score.

Reasonable and unreasonable uses

Reasonable: ranking a pile of claims so a person knows what to check first; seeing how text classifiers behave on a genuinely hard problem; a starting point for your own fine-tune.

Unreasonable: deciding whether something is true, moderating anything, or any decision about a person without a human reading the output.