How we test
Planned: every tool in a use case will run the same standard tasks in a sandbox, driven by AI agents. Testing started on 2026-10-16; only tools marked "tested" have scores, and thresholds are provisional (rubric v0.1). Some tools are tested as their open-source or self-hosted edition (for example Supabase self-hosted); this is labelled on the tool page and may differ from the hosted cloud product. We will record whether it worked, how long it took, what it cost, and screenshots.
Scoring (0–10 each)
| Outcome | 40% | Did the task work end-to-end? |
| Speed | 15% | Time to a working result. |
| Cost | 15% | Real spend for the task, plus free-tier generosity. |
| Developer experience | 15% | Setup, docs, errors, how much hand-holding it needed. |
| Reliability | 15% | Consistency across 3 repeated runs. |
Independence
Paid submissions and sponsorships never change scores or rank. Sponsored slots are always labelled. Tools are re-checked on a schedule; every page shows its last-checked date and sources.