How Claude Shannon used his wife as a language model in 1950
Grant Sanderson’s new 3Blue1Brown video starts from a 1940s question — how far can you compress text? — and arrives at why it matters for AI.
The bridge is from Claude Shannon’s information theory: “prediction and compression are mathematically equivalent,” two sides of the same coin. So a language model trained to predict the next token is…
