Every Time You Type a Sentence and Your Phone Finishes It, You're Running a 1948 Math Paper

When you type 'I'm going to the' and your phone suggests 'store,' you are running a 1948 math paper. Not a polished, recent, app-store-distributed math paper. A 77-year-old 79-page monograph, first published in the Bell System Technical Journal, by a 32-year-old mathematician named Claude Shannon. That paper is called 'A Mathematical Theory of Communication.' It is the most important single document in the history of digital technology, and almost nobody outside of electrical engineering has read it.

Predictive text, spell check, autocomplete, text-to-speech, machine translation, the compression that makes streaming video possible, the error correction that makes deep-space probes work, the entire modern practice of machine learning — all of it sits on top of Shannon's framework.

THE PAPER THAT INVENTED INFORMATION

“Information is the resolution of uncertainty. Every time your phone predicts your next word, it's reducing uncertainty, exactly as Shannon described.”

Claude Shannon joined Bell Labs during World War II, working on cryptographic systems for secure military communications. By the time he returned to unclassified research in 1946, he had been thinking about a deceptively simple question for years: what, mathematically, is information?

Before Shannon, information was a squishy concept. Engineers knew that telegraph, telephone, and radio channels had different capacities, but there was no precise way to compare them. There was no unit of information. There was no way to prove that a given message could or could not be transmitted through a given channel. Communication theory was, essentially, folk engineering.

“Shannon didn't just measure information. He gave us the tools to compress it, transmit it, and now, predict it.”

Shannon's 1948 paper changed that overnight. It defined information mathematically as the reduction of uncertainty. It introduced the 'bit' — binary digit — as the fundamental unit of information (the term was actually coined by Shannon's Bell Labs colleague John Tukey, but Shannon popularized it). It proved that any communication channel has a maximum capacity, measured in bits per second, and that messages can be transmitted reliably below that capacity regardless of how noisy the channel is, provided the right coding is used.

The paper introduced entropy as the quantitative measure of uncertainty. High entropy means unpredictability — like random coin flips, where the next outcome carries a full bit of information. Low entropy means predictability — like the letter after 'Q' in English, which is almost always 'U,' and therefore carries very little information. Entropy is measured in bits. English prose, Shannon calculated, has an entropy of roughly 1.0 to 1.5 bits per letter, even though the alphabet could in principle encode about 4.7 bits per letter (log2 of 26). The difference is the redundancy of natural language — the part that makes prediction possible.

WHY PREDICTIVE TEXT IS SHANNON APPLIED

The principle behind your phone's autocomplete is directly derived from Shannon. When you type a sequence of words, the system computes the probability distribution over possible next words, given everything you have typed so far. It then picks the word with the highest probability — or shows you the top three, as most modern keyboards do. That probability distribution is exactly what Shannon called the entropy of a source.

Early predictive text systems in the 1990s used simple Markov models: count how often each word follows each other word in a large corpus of text, and use those counts to estimate next-word probabilities. T9, the predictive text system that shipped on Nokia phones from 1999 onward, was a direct implementation of this idea. It let users enter words with fewer keypresses by exploiting the fact that, given a sequence of digits on a numeric keypad, there is usually only one likely word that matches.

Modern predictive text is the same idea at dramatically larger scale. Google's Gboard and Apple's keyboard use transformer-based neural networks — descendants of the same family of models that power ChatGPT — to predict the next word given a longer context. The model is much larger. The training data is much larger. The computations are different. But the framework, the question being answered — what is the probability distribution of the next word given the previous ones — is the same one Shannon posed in 1948.

SHANNON THE INVENTOR

Shannon was not a pure theorist. He built things. Theseus, a mechanical mouse he constructed in 1950, could navigate a maze, remember the path, and traverse it again faster on subsequent runs. It is frequently cited as one of the first machine learning systems. Shannon also built a computer that played chess (in a 1950 paper, he sketched what would become the core algorithm used by essentially every chess engine through the Deep Blue era), a machine that counted cards at blackjack (with MIT mathematician Edward Thorp), and a unicycle-juggling robot that he kept in his attic.

At Bell Labs, he rode a unicycle through the halls while juggling.

Shannon won the IEEE Medal of Honor in 1966, the Kyoto Prize in 1985, and the National Medal of Science in 1966. He died in 2001, after spending the last decade of his life with Alzheimer's disease — a disease that, tragically, eroded the memory of the man who had mathematically defined what memory and information were.

WHY IT MATTERS

Every act of digital compression, transmission, and prediction in the modern world runs on Shannon's framework. The ZIP file, the MP3, the H.265 codec, the WiFi protocol, the cellular radio in your pocket, the error correction code that keeps Voyager 1 in contact with Earth across 15 billion miles of space, the neural network that finished your last sentence — all of them are applications of information theory, with the fundamental concepts and units defined in a single Bell Labs paper in 1948.

Shannon's math is older than the transistor (1947's invention had not yet been commercialized when Shannon wrote the paper), older than the integrated circuit, older than the first programming language, older than the internet by a generation. It has outlasted every implementation built on top of it.

Your phone predicted your next word this morning. It used a 77-year-old equation to do it. The equation is still accurate. The math was right the first time.

(Sources: Claude E. Shannon, 'A Mathematical Theory of Communication,' Bell System Technical Journal, July & October 1948; Claude E. Shannon, 'Prediction and Entropy of Printed English,' Bell System Technical Journal, 1951; James Gleick, 'The Information: A History, A Theory, A Flood,' Pantheon, 2011; Jimmy Soni & Rob Goodman, 'A Mind at Play: How Claude Shannon Invented the Information Age,' Simon & Schuster, 2017; IEEE Information Theory Society historical archive)