About WikiCritic

A decentralized critical commentary layer for Wikipedia. These mockups illustrate the design of a system that aggregates public reviews, talkback, and AI assessments of Wikipedia articles—without hosting any content itself.

The Problem

Wikipedia is the most widely read reference work in history, but there is no independent, public mechanism for evaluating the quality, neutrality, or accuracy of its articles. Article subjects cannot correct factual errors about themselves. Experts have no visible channel for commenting on articles in their fields. Editors who attempt to improve captured articles face reverts, threats, and blocks—with no recourse outside the system controlled by the same administrators they are disputing.

The Concept

WikiCritic is a review aggregator—not a review host. It works like an RSS aggregator: reviews, talkback, and other commentary are published by their authors on their own platforms (personal blogs, Substack, X, academic websites, YouTube, Mastodon, etc.) and submitted to WikiCritic by URL. An AI agent fetches each URL, parses the content, and imports structured data according to WikiCritic's standards. The site never owns the data. Downstream developers can use or block sources at will.

This architecture solves the identity problem: the aggregator does not need to verify who a commenter is. X, Substack, university faculty pages, and YouTube already do that. All the agent needs to decide is whether to trust a particular transaction.

In addition to aggregating human commentary, WikiCritic generates an AI assessment of each article, based on a single revision. The AI assessment grades the article on a structured rubric—one independent call per criterion—and provides text reasoning for each grade. It does not adjudicate claims made by article subjects or other commenters—that is for the reader.

User Needs Served

  1. Quality assessment: The need of users to know whether an article is biased, whether it cites high-quality sources, whether it is complete, and what the issues might be with the Wikipedia accounts that govern it. This requires AI; there is no way for volunteers to do it systematically.
  2. Subject response: The need of the subjects of Biographies of Living Persons to respond to their own articles publicly, outside Wikipedia's control, with their corrections and objections on the record.
  3. Expert commentary: The need of experts to give feedback on articles in their areas of expertise.
  4. Right of reply: The need of anyone criticized in a WikiCritic comment—including Wikipedia's own editors—to answer it on the same page, rather than leaving criticism unanswered by construction.
  5. Suppressed pages: The need to document articles that were deleted, held in draft, or merged away, which by definition leave no article behind to comment on.
  6. Outside coverage: The need to gather third-party reporting and research about an article—court records, journalism, academic study—which nobody can license to us and which therefore cannot be a comment.
  7. Public talkback: The need of the general public to "talk back" to Wikipedia's admins on a given topic—without those admins being able to capture and control the process.
  8. Suppressed-edit documentation: The need of good-faith editors to document what was suppressed and why—to present the specific edit, the revert, and the case that the revert was wrong, with verifiable diff links.

The Consequence

Beyond all of that, there is an effect the aggregator produces simply by existing. A large body of poor grades, published where anyone can read them and attached to named articles, puts real public pressure on Wikipedia to reform. If the grades rise, the encyclopedia has improved and WikiCritic has helped bring that about. If they do not rise, WikiCritic becomes the standing, itemised evidence of how bad Wikipedia has become—or of how bad it always was, for those who doubt it was ever much good. Both outcomes are useful to a reader. Only one of them is comfortable for Wikipedia.

Mockups

The following pages illustrate the system across three examples: a topic article (Acupuncture), a Biography of a Living Person (Peter Schiff), and a real subject with no article at all (Jacob G. Hornberger). All pages are interlinked.

1
Acupuncture—Aggregator Page AGGREGATOR
The main WikiCritic page for a topic article. AI assessment, 7 reviews ordered by weight (experts first), a threaded rebuttal, 4 talkback comments, 2 related online pages, agent-assigned tags on every comment, the combined grade, EncycloSearch sidebar, and links to the full AI report and source documents.
2
Peter Schiff—Aggregator Page (BLP) AGGREGATOR
The main WikiCritic page for a Biography of a Living Person. Subject response (compiled from Schiff's real public statements) always appears first. AI assessment without adjudication of the subject's claims. Reviews, talkback with reverted-content / threat / block report tags, a threaded rebuttal from a dissenting commenter, 2 related online pages including the court judgment itself, and EncycloSearch sidebar.
2b
Jacob G. Hornberger—Aggregator Page (no article) AGGREGATOR
The third case, and a real one: English Wikipedia has no article on Jacob G. Hornberger, founder of the Future of Freedom Foundation and a candidate for the Libertarian presidential nomination in 2000 and 2020. Three deletion discussions, drafts speedy-deleted in 2019 for promotion and copyright violation, and both title variants creation-protected in June 2026. No assessment is generated because there is no revision to grade. Shows the real deletion timeline, three page-reverted-content reports, an administrator's rebuttal defending the deletion, a review of the redirect target, and the fact that Simple English Wikipedia does carry an article on the same subject.
2c
Source Documents: Jacob G. Hornberger SOURCES
Five comments as published: an academic note on the third deletion discussion, an administrator's Substack post rebutting it, a blog post from the editor who wrote one of the deleted 2019 drafts and says both deletions were correct, a terse X post about the redirect, and a review of the page that redirect points to.
3
Wikipedia + Extension Bar EXTENSION
A mockup of the WikiCritic browser extension as it would appear on a Wikipedia article. Compact sticky bar showing the grade, key criteria, and comment count. Expandable panel with a grade breakdown, condensed AI summary, and comment statistics. "View full WikiCritic page" button links to the aggregator.
4
Full AI Report: Acupuncture AI REPORT
Detailed text reasoning for each rubric criterion (Overall, Neutrality, Completeness, Overall Accuracy, Lede Accuracy, Style), based on a single revision. Identifies selective citation of the 2017 Cochrane review, the lede's unqualified "pseudoscience" claim, the compressed History section, and the disproportionate Adverse Events coverage.
5
Full AI Report: Peter Schiff AI REPORT
Detailed text reasoning for a BLP. Identifies the structural imbalance in the Euro Pacific Bank section (allegations foregrounded, legal outcomes buried), the selective gold-predictions timeline, the Daily Show as BLP sourcing, and the contested NYT quote presented without qualification. Does not adjudicate the subject's own claims.
6
Source Documents: Acupuncture SOURCES
Eleven comments as they appear on their native platforms (academic website, Substack, personal blog, X threads, Mastodon, Google Docs). Quality varies from polished academic reviews with full rubric to one-line tweets with no metadata. Shows what the AI agent would fetch and parse. The last carries no structure at all—nothing but prose and the required license line—as a test case for agent extraction.
7
Source Documents: Peter Schiff SOURCES
Ten source documents including Schiff's own website (styled after schiffradio.com), his X post, a Substack on BLP policy, a Google Doc reverted-content report with diffs, X threat and block reports, a measured financial journalist's review, a dissenting commenter grading the article B−, and a low-quality one-liner. Includes an academic expert review from a law professor.
8
Operator console: Assessments STAFF
The staff side of the site, and the only page here a visitor never sees. Search Wikipedia by title to queue an article at its current revision, then run the seven rubric calls when you choose to spend. Shows the monthly LLM spend against its cap, which articles have gone stale as their revision drifted, and what the run is doing while it runs. The title search is live: it queries the MediaWiki API as you type.
9
Operator console: Submissions STAFF
Everything the agent was asked to import and what it decided, searchable and filterable by state. Imports sit alongside refusals, each keeping the reason the submitter can read at their claim-token URL: no licence line, reviews six articles, target is a user page, login wall, thread rather than a single post. A separate Needs a human state collects what the agent must not decide alone — a legal demand, a suspected same-person submission, a possibly machine-written comment.

Architecture Summary

WikiCritic does not host reviews. It aggregates them from public URLs. Comments are tagged by the AI agent, which makes its own determination: Review, Talkback, Page suppression, Subject response, or Rebuttal. Talkback comments carry one or more of reverted content, threat report, and block report; Page suppression comments carry deletion report, draft report, or merge report. The agent additionally tags contents wherever they appear—factual correction, new sources, sourcing dispute, proposed remedy, commendation, omission, outdated, link rot, citogenesis, coordinated editing, undisclosed paid editing—plus two it observes rather than infers, diffs provided and sources cited. Systemic pattern is the one tag applied only when the author states it. Comments may endorse or dispute other comments by URL; there is no voting. Contributors are classified by the AI as Article subject (BLP only), Expert, or General public. Grades are optional and use a letter-grade rubric: Overall, plus six criteria—Completeness, Competency, Scholarship, Neutrality, Currency and Simplicity. Each may be given for the lede, the whole article, or both separately. The AI assessment is generated per-revision from the article text alone. Separately, the site reads Wikipedia’s public deletion, protection and revision logs—that is how a page removed from view gets a WikiCritic page at all, and how documented reverts are shown with elapsed times. A second submission path accepts related online pages—third-party coverage about an article, stored as link and excerpt only, with no grades and no tags. The site integrates with the Encyclosphere via EncycloSearch, showing other free encyclopedias' coverage of the same topic.