Triad is a three-layer benchmark intelligence service. We track every meaningful LLM, every meaningful agent, and the harness each one runs inside. Every score traces to a published source row. The cross-source aggregate is published weekly and audited continuously.
Score = cross-source mean across the upstream sources cited per row. $ / 1M = USD per million output tokens. Speed = output tokens/second when reported by the upstream. Δ wk = week-over-week change in cross-source mean. Layer-3 rows report Δ vs plain SDK, not absolute score. Source list at § Methodology · Sources.
Frontier and open-weights language models, scored across quality, speed, and price.
Agent benchmarks across coding, browsing, OS, and tool-use tasks. Includes product-mode agents.
Same model, same benchmark, different scaffold. The layer no public site measures.
Every cell on the scoreboard footnotes the upstream row and the retrieval timestamp. We never accept benchmark sponsorship — funded by subscription only.
Hourly for ArtificialAnalysis. Six-hourly for lmsys arena and SWE-bench. Daily for AgentBench, GAIA, terminal-bench, OSWorld. Weekly for HELM. Vendor blogs and paper PDFs are ingested manually on detection. Every fetch is logged with the source URL, the response hash, and the retrieval timestamp.
Canonical row: { model_id, harness_id, benchmark_id, score, score_kind, n, ci, source_url, retrieved_at }. Score kinds are not mixed across rows — pass-at-1 patch correctness, Elo, and quality-index points are stored separately. The aggregator reads the canonical table; the upstream is never consumed directly by a public surface.
When an upstream changes its scoring rubric, we re-ingest the affected history and surface a 'methodology bumped' footnote on every cell affected for 30 days. We never silently restate a previous week's number; if it changes, the change is footnoted.
Vendor blog posts are treated as sources, not as results. A vendor claim must be matched on at least one independent benchmark before it is allowed onto the cross-source scoreboard. The methodology page lists the verification rule and every cell links to the row that satisfied it.
The digest goes out Mondays 09:00 GMT. Every digest item is a single-sentence delta with a citation. Custom-slice subscribers can request on-demand cuts of the canonical table; turnaround is 24h. Every published number, in every channel, footnotes its upstream row and the retrieval timestamp.
The weekly digest, the cross-source scoreboard, and the archive. For the eval lead replacing a private spreadsheet.
Pay in USDT, USDC, or with a credit card on the NOWPayments hosted page.
Everything in Reader, plus four custom-slice requests per month delivered as PDF + CSV with a 24h SLA.
Pay in USDT, USDC, or with a credit card on the NOWPayments hosted page.
Single-tenant scoreboard URL, custom report cadence, named-payer invoice, methodology audit pack. Wire or USDT direct.
Reply received within 48h with a single-tenant pilot URL.
If your question isn't here, the methodology page covers everything in long form, or email founder@prin7r.com.
ArtificialAnalysis covers Layer 1 (LLMs) extremely well. Triad covers Layer 1 plus Layer 2 (Agents) and Layer 3 (Harnesses), and we cross-source against multiple upstreams including ArtificialAnalysis itself. We cite them when their data feeds a row.
HELM is rigorous but cycles every 4–8 weeks. We surface deltas hourly. We use HELM as one of our weekly sources rather than as the only source.
We aggregate from a fixed list of upstreams: lmsys arena, ArtificialAnalysis, HELM, SWE-bench, AgentBench, GAIA, terminal-bench, OSWorld, plus vendor blogs and paper PDFs. The methodology page lists every source. Every cell on the scoreboard footnotes the upstream row and the retrieval timestamp.
We pin a source_version on every ingested row. When an upstream bumps its methodology, we re-ingest the affected history and surface a 'methodology bumped' footnote on every cell affected for 30 days.
Not in Wave 2. We aggregate published benchmarks. If we ever decide to run our own — to fill a gap — we will publish the methodology and the conflict-of-interest implications first.
We accept zero compensation from any model lab, agent vendor, harness vendor, or benchmark organisation. Funded by subscription only. Published as a permanent page on the site.
Reader and Custom-slices: cancel any time, pro-rated refund on unused days within the same billing month. Enterprise: per signed contract.
NOWPayments hosted invoice converts at the merchant level — readers can pay with USDT, USDC, or with a credit card on the NOWPayments hosted page. Enterprise tier accepts wire transfer or USDT direct invoice via founder@prin7r.com.