Editorial
How We Test and Rank AI Tools
One shared task per job, the same inputs for every tool, and a written record of what broke. This page is the method, so you can judge whether the verdicts are worth anything.
The shared brief
Every tool we compare on a given job gets the same brief. For lesson planning that means the same grade band, the same standard code, the same time limit and the same output format. For feedback on writing it means the same anonymised student sample and the same rubric.
A tool is never judged on a hand-tuned prompt written to flatter it. If a tool needs prompting skill to produce usable output, that is part of its score.
What counts as a failure
- Invented standards codes. A confidently wrong code is worse than no code.
- Output that cannot be used without rewriting. If more than half the text needs replacing, the tool did not do the job.
- Free-plan limits that break a normal week. A quota that runs out on Wednesday is a real failure, not a footnote.
- Student-data risk. Any tool that requires student PII for a core task is marked down, whatever its output quality.
How scores are set
Ratings are out of five and reflect the same weighting everywhere:
- Task quality (50%) — did the output need a teacher to fix it, and how much?
- Honest free tier (20%) — what a teacher can actually do without paying.
- Time to a usable result (20%) — including prompt overhead and setup.
- Privacy posture (10%) — whether core tasks need student data.
A tool with no affiliate programme is scored on the same four criteria. Commission is not one of them.
What we cannot verify
We do not have admin access to any vendor’s backend. We cannot confirm how a vendor stores data beyond what they publish, we cannot see their model, and we cannot guarantee a standard alignment their own documentation does not state. Where a vendor claims FERPA, COPPA or GDPR compliance we say “the vendor states” and link to their policy — we are not a certification body, and neither is a badge on a pricing page.
Pricing
Prices are read from the vendor’s own pricing page and dated. Until a person re-reads that page, a price on this site is labelled indicative. We never estimate a price we have not seen. Where a vendor quotes districts privately, the table says “quote-based”.
What we update, and when
- Prices and free-tier limits: re-checked monthly where a tool is ranked in a pillar page.
- Verdicts: revisited when a vendor ships a significant change.
- Corrections: any reader can tell us a price is wrong; we fix it and date the change.
How many tools we cover
51 tools have a full profile today, and the pillar guides name every tool that made the shortlist. The catalogue is deliberately smaller than the directories that list 150 products, because each entry here carries a verdict, a dated price and a record of where the tool breaks.
About the writing
Articles are drafted from structured tool data, prompt templates and testing notes, then edited by the writer named on the page. Every claim that could not be verified is labelled as such on the page. If a page says “we tested”, the task and the failure are described in that page.
See also: editorial policy and the people who write here.