The Math Professor Who Cost Intel $475 Million
Thomas Nicely was checking primes when his Pentium gave him a wrong answer. Five missing entries in a lookup table did it.
On June 13, 1994, Thomas Nicely added a Pentium to his prime-number research at Lynchburg College and noticed his sums of reciprocal twin primes were drifting. He ruled out his code, then his compiler, then his other machines. On October 24, he wrote to Intel.
The bug lived in the floating-point divider. Intel had replaced the 486 slow division with the SRT algorithm, named for Sweeney, Robertson and Tocher, which spits out two bits of quotient per clock cycle by reading from a 2,048-entry lookup table. Five of those entries, all the value +2, never made it to the lithography masks. The cells in the actual silicon contained zero. Most divisions still came out right; a sliver of inputs hit the missing rows.
The famous test case was 4,195,835 divided by 3,145,727. The correct answer is 1.333820449136241002. The Pentium returned 1.333739068902037589, wrong from the fourth digit on.
Intel knew about the flaw before Nicely's email and was quietly shipping fixed steppings. When the story broke in Electronic Engineering Times on November 7, the company first told customers their workloads were unlikely to ever hit it. IBM ran the math, claimed an error every nine days for a typical spreadsheet user, and stopped shipping Pentium PCs in mid-December. The press cycle that followed is why Intel offered no-questions-asked replacements on December 20.
On January 17, 1995, Intel took a $475 million pretax charge against earnings, its first chip recall, and roughly a billion in today's dollars. Nicely never got a bounty. He kept finding twin primes.
Make Recess yours.
Sign in to save the ones you loved, never see the same thing twice, and tell us what you want more of.