Collapse Index Labs · Alex Kwon

hello. this is my research page. 🤓

i research the invisible structures inside ai models, especially the ones that make my head hurt.

i developed the collapse index after something in my life became stable, unresolved, and wrong.

then i ablated the mechanism my metric was tracking.

then it broke. 😔

now my research follows this lens:

"collapse happens when a system preserves the thing that sounds final and loses the thing that could make it true again."

hmm... 🤔

if (truth && !alignment && collapse) {
    transmit(signal);
}

🏺 🦖 🌙 🧬 ❤️

— Alex Kwon

Tools

dinostompa verification layer for AI evaluations. eval scores get published; the instrument that produced them almost never gets checked. dinostomp is that check, across the whole pipeline: the dataset, the scorer, the runs, whether the number survives its own sampling noise, and whether the claim has the evidence it needs. pointed at MMLU, it found a subtraction item keyed to two identical correct options. the error class was already known from MMLU-Redux, which hand-annotated 5,700 items; run against those annotations, dinostomp found two more double-keyed items the annotators had marked ok. prior art is cited in the ledger. it also audits agents: their tool traces are recorded by the harness rather than self-reported, and it can withhold an agent's retrieved evidence to ask whether the answer actually depended on it.Alex Kwon · 2026 · every check negative-tested against a planted defect · code
moons-dont-talka preflight gate for ML training runs. moons don't talk; receipts do. before the GPUs get a job, it reads the data, the config, and the machine, runs a short canary of the real training command, proves the checkpoint by restoring it in a fresh process, projects runtime and cost against the limits you declared, and writes a receipt that binds the verdict to hashes of the data, config, code, and environment. change any of them and the verdict is invalid, field by field. any trainer that can honor five environment variables can be gated; the hugging face adapter has run on cpu, a t4, and an a10g. its first day of live tests produced six ledger entries against itself, including a disk check that passed on eight exabytes.Alex Kwon · 2026 · sibling of dinostomp: same method, different cargo · code

Case Studies

Two Tenths of a Percent - Pretraining Experimenta pre-registered experiment: what dinostomp-style filters do to a small pretraining run, measured against seed noise. four training sets cut from 160,000 FineWeb documents at one 40M-token budget (raw, deduplicated, dedup plus hygiene, dedup plus 500 planted WikiText-103 test paragraphs), twelve 30M-parameter GPT runs, three seeds each, five predictions committed to a git hash before the data was pulled. four held, one failed and is published. planting 0.2% of the tokens cut the benchmark perplexity 44% while the training loss and the in-distribution held-out sat flat; dinostomp flagged 499 of 499 planted paragraphs with zero false hits; seed noise on the out-of-distribution eval was 6 to 13%, wider than dedup or hygiene moved it. dedup did nothing because FineWeb already did it, and the write-up says so.Alex Kwon · 2026 · $2 of GPU · code + findings
The Bar Was Seasonal-Naive - Data Analysisan audit-first sales analysis and demand forecast of Online Retail II (UCI; 1,067,371 invoice lines of a UK online retailer, 2009-2011). the audit finds a nine-day byte-identical overlap between the file's two sheets, admin fees posing as products, and warehouse write-offs inside the sales table; the forecast's rules were registered before the data was opened, and nothing beat seasonal-naive (MAE £52,948, 20.0% MAPE on a 13-week Christmas-ramp holdout). published as the finding, not a failure.Alex Kwon · 2026 · report · code + findings
Three Rulers, One Dataset - Data Analysisan audit of MovieLens 25M, one of the most-used public research datasets in the world: 25,000,095 ratings across 25 years. every structural promise its documentation makes holds exactly, and the timeline underneath is three rating instruments stitched together at seams the documentation never mentions. the oldest instrument flattered bad movies by nearly a full star. the data is immaculate; the ruler moved.Alex Kwon · 2026 · report · code + findings
July Orders Review - Data Analysisan end-to-end analyst engagement on a deliberately dirty e-commerce dataset: a paste-over that both duplicated one order and destroyed another, a region spelling that silently split every regional rollup, and a sales channel that returned 70% of the revenue it booked. the write-up separates what the data shows from what it can prove, and maps every open question to the record that would settle it.Alex Kwon · 2026 · report · code + findings
The Join Is Dirtier Than the Data - Data Analysisan audit-first analysis of the Animal Crossing: New Horizons catalog (30 tables, 18,962 rows). the tables are close to immaculate; the string keys between them are not: one capitalization mismatch silently drops three villagers from any song join and manufactures a false finding, three placeholder rows inflate the song catalog, a sentinel string poisons the price column's type in 22 of 25 tables, and a perfect personality-gender partition makes every gender claim a design claim. includes one false alarm by the analyst, corrected and kept on the record.Alex Kwon · 2026 · report · code + findings

Essays

Fifteen Questions Before I Trust a Numbera plain-language guide to evaluation fundamentals: the fifteen questions to ask of any benchmark score before believing it. what exactly is measured, is the metric a proxy, did the model see the answers, compared to what, how much does it wobble across seeds, which component is doing it. each question carries a receipt from published work (the planted-contamination experiment, the option-order paper, the filed MMLU defect report), and the page ends with two worked examples, MMLU from the auditor's side and Reclaim Evaluation from the builder's, and a plain-language glossary.Alex Kwon · 2026 · essay · read
Evals Are a Supply Chain Problema program management essay on AI evaluation pipelines, written in the dialect of an 11-year consumer-goods supply chain: shards are pallets sized by affordable damage, idempotency is one BOL slot per pallet, dead shards are red-tagged pallets in the hold cage, and a dashboard that hides its shortages is a distributor scorecard with a secret baseline. pin the definitions before you do anything.Alex Kwon · 2026 · essay · read

Selected Publications

★ If you read one, read this one.

Coherent Values, or the Frame That Asked? From Preference Transitivity to IdentifiabilityAlex Kwon · 2026 · Apart Research
Explicit, Not Longer: What Makes Epistemic Stance Survive Memory CompressionAlex Kwon · 2026 · arXiv:2608.06953 · code
FACTWASH: Catching AI Rewrites That Wash Hearsay into FactAlex Kwon · 2026 · arXiv:2608.03372 · code
Every Signal We Found Was an Artifact: A Calibrated Control Battery for Secret-Loyalty AuditingAlex Kwon · 2026 · Apart Research sprint · code
Breaking Refusal in the First Half: A Mechanistic Study of the Prefill JailbreakAlex Kwon · 2026 · arXiv:2607.14147
They Infer What You Meant: Models Represent Communicative Intent More Reliably Than They Act On ItAlex Kwon · 2026 · arXiv:2607.03598
Manufactured Confidence: How Memory Consolidation Turns Hearsay into Confident FactsAlex Kwon · 2026 · arXiv:2606.29279
Reclaim Evaluation: A Lossy Memory Is Worse Than an Empty OneAlex Kwon · 2026 · arXiv:2606.25449
When Context Flips, Safety Breaks: Diagnosing Brittle Safety in Aligned Language ModelsDasol Choi, Alex Kwon · 2026 · arXiv:2605.27851