Strength is how sure the evidence is: a confidence interval that clears chance, how large the sample is, and whether the measure is something people did or only something they pressed. AUC 0.5 is chance. A "press" is a like, an upvote or a verdict button.
| # | Feature | Evidence | Measure | Strength |
|---|---|---|---|---|
| 1 | Written for the customer, not our tools: customer and money words up, tooling words down | Customer words AUC 0.62 (CI 0.56 to 0.68). Tooling words AUC 0.34 (CI 0.27 to 0.41), so fewer is better. Every devlog paragraph about our tools was not good (0 of 22). | Mackenzie's 172 paragraph verdicts | Strong |
| 2 | Close to writers our customers read (Housel, Graham, Fishkin, McKenzie) | AUC 0.67 (CI 0.58 to 0.75). Closeness to the HN/tech set runs the other way: AUC 0.36 (CI 0.27 to 0.44). | Mackenzie's 172 verdicts | Strong |
| 3 | Shorter paragraphs | Paragraph length AUC 0.38 (CI 0.30 to 0.46), so shorter is better | Mackenzie's 172 verdicts | Moderate |
| 4 | Threading: questions open throughout, all closed at the end | Mid-article, the open count stays above zero in nearly every article; that half is the floor everyone clears. The edge is the other half: reaching zero at the end. The same writer's or newsletter's hits answer all their open questions by the end; the low posts answer about 77%. AUC 0.65 on HN authors (CI 0.50 to 0.81), 0.67 on Substack (CI 0.50 to 0.88). Goodfire's "Already Clean" closes its main thread at about 70% of the way through (the title question is answered there), then runs an appendix of open problems, each one answered where it's raised. | HN upvotes and Substack restacks plus comments (presses); Mackenzie's judgement (Goodfire) | Moderate (CI touches chance) |
| 5 | A concrete named loss in the article | HN same-event pairs: 4 of 15 higher-voted vs 0.2 lower-voted. Within the 5 HN authors: 8 of 10 hits vs 5 of 10 flops. | HN upvotes (presses) | Moderate (noisy marker) |
| 6 | Close exits: show the obvious fix failing | Within the 5 HN authors: 9 of 10 hits vs 6 of 10 flops | HN upvotes | Weak |
| 7 | Many questions open at once (pillars in the nut graf, threading) | All 3 Longform best-of pieces above all 6 ordinary picks: 5.7 vs 2.6 open on average. Goodfire's "Already Clean" opens 12 questions in its first section; good articles average 2.9. | Editors' best-of picks; Mackenzie's judgement (1 article) | Weak (n = 9, plus 1) |
| 8 | More stakes levers, within one writer | 4.7 vs 3.6 levers, hits vs flops (AUC 0.66, CI 0.38 to 0.90) | HN upvotes | Weak (CI crosses chance) |
| 9 | Headline word buckets (my buckets, labelled after the fact) | Time words 19 to 4 (p = 0.003), and 8 of the 19 are video length ("2 minutes", "30 seconds"). Negative or intense words 11 to 3 (p = 0.06), half of them "never", "worst", "wrong" rather than losses. A number 16 to 8, superlatives 14 to 8, "you" 19 to 13 (all p > 0.1). Six buckets were tested, and only the time bucket would survive a correction for multiple tests. Cause unknown: likely recency, a low time cost, or novelty, not a deadline. | Upworthy clicks, 100 tests from 2013 to 2015 | Weak |
| 10 | A draft close to reader-rewarded writing (centroid) | On HN same-event pairs, the winner sat closer to the other winners in 13 of 18 pairs (10 of 15 events) | HN upvotes | Weak |
Checks that run today: bin/ladder.ts checks rows 1 and 4. bin/ref-score.ts with evals/refs/customers checks row 2.
| Feature | Result |
|---|---|
| Sentence length, numbers, "you", "I/we", backticks, proper nouns in paragraphs | Near chance on Mackenzie's 172 verdicts (AUC 0.45 to 0.52) |
| Closeness to Mackenzie's own good paragraphs (a "taste centroid") | AUC 0.56 (CI 0.47 to 0.65) |
| Reversal words in headlines | 11 to 11 |
| A clock (deadline) as a lever | Not shown. The headline result was time words, mostly video length, labelled "clock" after the fact (Mackenzie, 12:09: "naive overfitting"). No separation in full articles. |
| Stair steps, odds open, near-win reversal in full articles | No separation in 41 articles |
| Lever count across mixed sets | AUC 0.49; genre swamps it |
| Questions opened in the first chunk, across all sets | AUC 0.51 (it separates only within Longform and Goodfire) |
| HN/tech references as a target | Predicts not good (AUC 0.36); retired |
| HN Popularity rank as a guide to X | The authors rarely write X Articles; engagement per follower doesn't follow HN rank |
| Automatic topic references from HN search | Returned product pages for business topics |
| Unvalidated ladder checks: something to claim, headline about our tool, first screen, sections end open, ending asks for action | Posts Mackenzie liked fail some of them, so they're probably miscalibrated |
| bin/threads.ts on local Qwen | Rarely marks a question answered; unvalidated |
| The v1 payoff result (AUC 0.67) | Withdrawn: an artifact of the old marker |
| prose-scrub | Removed as a gate (Mackenzie, 07:21) |
An article is a trade. The reader pays with attention, and risks wasting it. In return they want a gain: something they can do or know that they couldn't before, or the reassurance that they're on the right track. The writer wants something too. For us, that's trust, and then a message to Mackenzie.
The reader never decides once. They decide again at every line: is what comes next worth more to me than the best other thing I could do right now? Two forces keep the answer yes:
Great articles thread the two. They open several questions early, and the count of open questions never reaches zero until the end. A reader who holds no open question has no reason to read the next line, so they leave. A section is only worth writing if it pays a question off, opens one, or both, for this reader.
My last page listed sections and advice as if each counted the same, and it didn't mention pull at all. That's wrong on both counts, and this page fixes both. My first version of this page also described threading as closing one question and opening the next. That's alternating, which is a different and weaker craft.
Open loops: the urge to finish. Zeigarnik (1927) reported that people remember interrupted tasks better than finished ones. That memory effect mostly fails to replicate. A 2025 meta-analysis found no memory advantage for unfinished tasks. But it did find a general tendency to resume interrupted tasks, the Ovsiankina effect. So an open question doesn't keep a reader by sticking in memory. It keeps them by creating an urge to go back and finish. A writer opens a loop on purpose and closes it later.
Curiosity gaps. Loewenstein (1994): curiosity is the feeling of a gap between what you know and what you want to know. It's strongest when the reader knows enough to see the gap and feel close to closing it. Curiosity also switches on the brain's reward circuitry and improves memory for the answer (Kang and others, 2009). On WeChat, curiosity-gap headlines won clicks and lost likes. In a feed, people mostly do one or the other, not both. A like is often an acknowledgement: I saw it, I agree, my friend posted it. That matters more where acknowledgement rituals are rich, as in many Asian languages. So the finding says people strongly prefer to click on a gap. It also says a press of "like" is not the same as liking something.
Cliffhangers, from the serials. Dickens published novels in monthly parts, starting with The Pickwick Papers in 1836. Each part ended on an open situation so readers would buy the next. By the 1860s this was standard in serials, and TV later did the same at every ad break. The article version: end a section on the question the next section answers.
Threading. The writer opens several questions up front, in the nut graf, and keeps the count of open questions above zero until the very end. YouTube scripts do this by naming the video's pillars at the start ("the two or three things this video covers"). A viewer who has been promised three things and has seen one still has two reasons to stay. MrBeast's production handbook builds whole formats on one big question held to the end: in "last to leave", viewers stay to see who leaves the circle last. On top of that come re-engagements at minutes 3 and 6, "stair stepping" (each stake bigger than the last), and a rule never to signal the end except to build toward the payoff.
This is different from alternating. Alternating closes one question and opens the next, so at every handover the reader holds exactly one question, and sometimes none, and those are the moments they leave. Threading runs strands in parallel (the A story and the B story), so when one pays off, others are still open. Only the ending closes them all, and then it offers the next step.
The check: count the reader's open questions after each section. If the count reaches zero before the last section, the piece has a hole where readers leave. If questions are still open at the end, those are unpaid promises. A local model now reads drafts section by section and keeps this count (bin/threads.ts). It's unvalidated: it counts questions opened well, but often misses when one is answered.
Setup and payoff (Chekhov's gun). Anything shown early has to be used later, and anything never used should be cut. A setup is a promise. Its payoff is the reward that makes readers trust the next setup.
Causal links: but, therefore. Events joined by "and then" give no reason to read on. Events joined by "but" (a problem) and "therefore" (a consequence) carry the reader from each beat to the next.
Starting in the middle (in medias res, the narrative hook). Open on a striking moment and explain how it happened afterwards. "I once accidentally bought a horse." The reader holds the question "how?" from the first line.
The slippery slide (copywriting). Sugarman: the job of the first sentence is to get the second sentence read. The headline sells the subhead, the subhead sells the first line, and so on down. Short bridges between sections ("Here's the catch.") keep the slide going. Copywriters call them bucket brigades.
A metric records what someone did, not what they meant. A like can be an acknowledgement. A vote can be agreement with the crowd. A stated preference is something people say, and it can differ from what they choose. So for every number below I ask what the person actually did, and what that behaviour shows.
I checked the ladder for this mistake. Three of its sources are presses, not verdicts:
The behaviour sources hold up as behaviour: clicks (Upworthy), time on page (Chartbeat), task results (Nielsen), and resuming interrupted tasks (Ovsiankina).
Each step only matters once the step below it holds. A reader who never starts never sees your evidence, and polished prose can't save a piece nobody wants. So fix the lowest broken step first. Here's the order, and why each step sits where it does.
1. A real reader with a real want. The piece is for someone who exists and is in a situation they want to get out of. Why first: this step decided your 172 paragraph verdicts almost alone. Every paragraph from our devlogs about our own tools was not good (0 of 22 across four posts). Most paragraphs from pages for business owners were good: all 16 on the two E-E-A-T posts, and 15 of 18 on where small business owners hang out. Features of the prose itself (sentence length, numbers, "you") predicted your verdicts at about chance.
2. A payload worth copying. The reader leaves able to do or know something that puts them ahead of their peers. Why second: on Hacker News, the posts that got more upvotes than others on the same event were by someone who did the work and listed concrete findings. Upvotes are presses, not verdicts, so treat this as what people chose to push up. Writing quality and the outlet's prestige did not decide it. Without a payload, every later step is decoration.
3. A headline that carries the payload. It promises that gain, to that reader, specifically. Why here: the headline is the only part most people see. Ogilvy: five times as many people read the headline as the body. In the 100 Upworthy tests we saved, the same story's better headline got a median of 2.4 times the clicks.
4. A first screen that pays or opens a loop. It either delivers the promise at once (when the reader arrives with a question) or opens a question too good to leave (when they arrive with none). Why here: Chartbeat found about 55% of pageviews got under 15 seconds of attention. In our own pairs, answers that made readers wait got fewer votes, and stories that opened a question got more. Votes are presses, so this shows what people chose to reward.
5. The pull engine. Several questions opened in the nut graf, an open count that never reaches zero until the end, but and therefore, cliffhangers at section ends, setups paid off. Why here: it decides how far a reader who started gets. It can't make anyone start, so it sits above the opening, and it does more than any line-level fix, so it sits below them.
6. Belief. Concrete specifics, names, numbers, a real moment, limits stated once with the fix. Why here: readers need belief to act, not to keep reading. Most people judge by fluency, confidence and specificity, not by checking. So belief comes from being concrete, not from hedging.
7. Low decoding cost. Plain words, short paragraphs, headings a scanner can enter by, active voice. Why here: it cuts the cost at every step but can't add value. Nielsen's concise version tested 58% better. In your verdicts, shorter paragraphs did better, while sentence length didn't matter.
8. Payoff and the next step. Close the loops you opened, then give one action: try this, install this, write to me. Why here: only readers who got this far are still around, and they're the most willing. For us, this is where the trade pays Mackenzie.
9. Polish. Rhythm, word choice, a voice. Why last: it's what readers notice least. Burstiness and stock-phrase counts, the measures prose-scrub used, don't predict your verdicts.
My devlogs failed at step 1, and I spent my effort on steps 7 and 9. That's why they read as competent and dead. From now on, I check the ladder from the bottom up and fix the lowest broken step before touching anything above it.