An evidence assistant that checks every quote before it answers.
Answers questions about public NNPC information with local open-weight models. Every claim carries a quote that code checks word for word in its source, figures are checked again by a second model, and unsupported questions get a clear refusal.
Runs locally; a demonstration is available on request.
- Sector
- Energy
- Method
- Local LLMs and RAG
- When
- Oct 2026
- Context
- Personal project, independent of NNPC
- Role
- Solo: collection, document reading, search, answer checks, API and web app
Key figures
- 4 local open-weight models answer each question; no cloud calls
- 0 claims shown without their quote found word for word in the source
- 100% test coverage, enforced on every check
How it works
- Collect : Obeys robots.txt; files that bar crawlers come by hand
- Read pages : As a browser lays them out, headings kept
- Read documents : PDF layout and tables, spreadsheets, scanned pages
- Gate : Each reader passes an accuracy test before indexing
- Index : Postgres vector and keyword search, plus a small graph
- Search : Vector and keyword search over the index
- Rerank : A cross-encoder orders the passages
- Write : Gemma 4 writes short claims, each with a quote
- Check : Code finds every quote, number and date in its source
- Judge : A model of another family checks every figure
- Answer : Verified claims with dated sources, or a refusal
Architecture
- A model runs here
- Deterministic code
- An outside source
- A stored artefact
- A boundary the design refuses
-
Read once, answered with checked quotes
Public pages and documents are collected within each host's rules, read by the reader that fits each kind, and indexed only after that reader passes an accuracy test. For each question, four local model servers search, write and judge: Gemma 4 writes short claims with quotes, code checks every quote, number and date word for word, and Nemotron 3 Nano, from another model family, checks every figure. Nothing collected leaves the machine.
Open full size: Read once, answered with checked quotes
The question
Public information about a large company is spread across web pages, press releases, annual reports, scanned PDFs and spreadsheets. Can local open-weight models answer questions about it with every claim traceable to its source, and say plainly when the sources do not answer? PetroKnowledge starts with NNPC Limited. It is independent: not affiliated with or endorsed by NNPC, and it uses no company logos.
Collecting within the rules
- The collector obeys each host’s robots.txt, including its crawl delay, sends at most one request a second to a host, and never follows a redirect off the company’s allowed hosts.
- Many reports are served from hosts whose robots.txt bars every crawler, so the collector never fetches them. A person saves those public files by hand, or lists them for a slow, named download, and each file records how it arrived.
- The company’s terms allow no copy beyond personal use, so collected content stays on the machine that collected it. The public repository holds the code and none of the content, and the assistant runs as a private demonstration.
Reading what the sources actually say
Web pages are read the way a browser lays them out, keeping the headings above each line. Some pages ship most of their text as data that the browser assembles; one profile page holds about 6,000 words that way and far fewer in its HTML, so such pages are read from the data. Rendering every page in a headless browser was tried and dropped: it took 30 to 54 seconds a page to get text that was already collected.
PDFs are read for layout, reading order and tables, and spreadsheets keep every figure with its row label, column and unit. Many reports are scanned pictures. An OCR model transcribes them, but no transcription passed its accuracy test, so transcripts help search find a page and are never quoted; the answer points to the page instead.
Answers that can be checked
Each answer is a list of short claims. Code checks that every claim’s quote appears word for word in its source, along with any number, amount, date or regulation it states. A claim that fails is removed, never repaired. A second model from another family, NVIDIA’s Nemotron 3 Nano, checks every claim that states a figure against the passages it cites.
If the checks leave part of a question unanswered, the reply says which part. Requests for investment, legal or safety advice get a fixed referral. A small knowledge graph, built by code rather than by a model, tracks the business units and who holds which office. The current holder is decided by date, not by how often a name appears, because former office-holders are mentioned far more often than current ones.
How it was tested
- Every reader was tested before indexing. The truth was written from page images before any reading was looked at, each test set was registered by its SHA-256 hash, and a fix was kept only if it held on pages it was not built from.
- Answers are graded against question sets written independently of the assistant. Held-out sets are asked once per release, and a change is kept only when it beats the noise measured by asking the same set twice.
- The code runs at 100% test coverage. The web app is tested for WCAG 2.2 AA in a real browser, in light and dark.
Limits
- The content cannot be republished, so the assistant runs on its owner’s machine and is shown on request. The test sets stay private with it, which is why this page quotes no accuracy figures.
- Scanned pages can be found but not quoted until a reading of them passes its accuracy test.
- It covers NNPC so far; the Dangote refinery and Nigeria LNG come next.
- It needs four local model servers running: the embedder, reranker, writer and judge.