The bug that only showed up on 1 in 4 page loads

Background

bopomofokids.com teaches each of its 37 Zhuyin symbols alongside an example word — ㄈ pairs with 飛機 (airplane) ✈️, for instance. Rather than hard-coding one word per symbol, each symbol carries a small array of candidate words, and getWordVariant(item) picks one at random the first time a symbol is shown, then caches that choice for the rest of the session. Reload the page and you might get a different word for the same symbol; within one session, it stays consistent across every mode.

There’s one hard rule that makes this safe: every word in the dataset is supposed to be unique. No word is meant to appear as a candidate under two different symbols. That rule wasn’t actually being checked anywhere — it was just an assumption everyone adding new words was expected to hold in their head.

The bug

ㄈ’s word list included 飛機 (fēijī, airplane). So, at the time, did ㄟ’s — because 飛機’s second character legitimately contains the ㄟ sound (f-ei-jī), and both symbols’ candidate lists had only two words each. Both were valid choices individually. Nobody had checked whether the same word showing up under two different symbols was a problem, because on any single page load it usually wasn’t visible at all.

QuizMode draws its wrong-answer choices partly from other symbols’ resolved example words. When a question involving both ㄈ and ㄟ came up and both symbols happened to have resolved their random pick to 飛機 in that session, the quiz rendered two choice cards showing the identical ✈️ emoji and the identical word — a child (or a QA pass, or a screenshot) had no way to tell which card was the intended answer.

Why this hid for a while

With two candidate words per symbol at the time, and each symbol’s pick made independently, the odds that both ㄈ and ㄟ landed on their shared word (飛機) in the same session were:

P(ㄈ picks 飛機) × P(ㄟ picks 飛機) = 1/2 × 1/2 = 1/4  →  ~25% of loads
      the actual odds
    

A quarter of the time isn’t rare — but it’s exactly the kind of probability that survives casual manual testing. Reload the page a couple of times while eyeballing the quiz, see it look fine both times, and move on: the chance of missing a 25%-likely bug across two independent checks is 0.75 × 0.75, or roughly 56%. You’d need to reload it around ten times in a row before the odds of never seeing it dropped under 6%. Nobody reloads a page ten times to manually re-check a quiz screen that already looked correct the first two tries.

The fix

The immediate fix was narrow: swap ㄟ’s first candidate word from 飛機 to 杯子 (bēizi, cup) — still phonetically correct for ㄟ, no longer shared with ㄈ. That made the specific collision impossible, not just less likely.

The real fix was broader: the “every word is unique across the dataset” rule became an actual documented invariant instead of an assumption, with an explicit instruction to grep the full word list for a candidate word before adding it under a new symbol. Around the same change, each symbol’s candidate list grew from two words to three — which incidentally lowers the odds of any future accidental collision surviving unnoticed, but that’s a side effect, not the fix. The fix is that the collision is no longer allowed to exist in the data at all.

Takeaways