note: embedded demo at the end

Fact Check LIAR

A fact-checking system that retrieves similar political claims, predicts a six-way veracity label, and generates a readable verdict.

links

built with

LIAR Dataset
BM25
FAISS
BERT
Ollama
Streamlit

FactCheckLIAR is a retrieval-augmented fact-checking app built on the LIAR dataset. Given a user claim, it searches for related statements using a hybrid sparse+dense retriever, predicts the claim's truthfulness with a fine-tuned BERT classifier, and turns the result into either a concise verdict or a detailed evidence view. I recently extended the project with persistent indexes, optional Ollama-based response generation, a template fallback, and a more practical CLI/Streamlit workflow.

Dataset

LIAR

12.8k manually labelled statements
6 veracity classes
Speaker, party, context, and historical truthfulness metadata

Updates

index persistence

Saves FAISS and BM25 indexes to a local cache/ directory.
Uses a dataset hash to invalidate stale indexes when data/train.tsv changes.
Cuts repeat startup time from about 30-60 seconds to about 2-3 seconds.

response generation

Adds optional local LLM responses through Ollama, configured with OLLAMA_API_URL and OLLAMA_MODEL.
Falls back to deterministic template responses when Ollama is unavailable or disabled.
Supports concise answers and verbose evidence-focused explanations.

Labels

pants-fire

completely false, no factual basis

false

factually incorrect, not outrageous

barely-true

some truth, but misleading

half-true

partially correct

mostly-true

mostly correct, minor inaccuracies

true

completely true

The system is organized as a small pipeline: retrieve similar claims, classify the query, then generate a user-facing answer.

Retrieval

Claims rarely appear in exactly the same wording as the dataset examples, so the retriever combines lexical and semantic similarity. BM25 handles keyword overlap, while FAISS searches dense vectors produced with all-MiniLM-L6-v2. The two scores are normalized and fused with a weighted sum before selecting the most relevant supporting claims.

Sparse index

BM25 tokenizes the LIAR statements and ranks them by term relevance, which keeps exact political terms, names, and phrases influential in the result set.

Dense index

FAISS stores normalized sentence-transformer embeddings so the app can find semantically similar claims even when the wording changes.

Classification

The classifier is unshDee/liar_qa, a bert-base-uncased sequence classifier fine-tuned for the six LIAR labels. The app loads the model from a local classifier_model/ directory when available, otherwise it downloads the saved model from Hugging Face. The CLI can still train from scratch with --train_classifier.

Interfaces

command line

--query accepts a claim directly from the terminal.
--verbose shows retrieved evidence and context.
--no-llm forces template responses for reproducible output.

streamlit app

Text input for claims.
Toggle for detailed evidence.
Toggle for LLM-based response generation.
Resource caching with st.cache_resource.

Demo

Try claims that require both retrieval and classification:

Is it true that Barack Obama was born in Kenya?
Is it true that climate change is a hoax?
Is it true that increasing the minimum wage will lead to massive job losses?

notes

too bright? click ↝