113 years

Andrey Markov took 20,000 letters from Pushkin’s Eugene Onegin and classified them as vowels or consonants. Then he counted how often one kind followed another.

The order mattered. Knowing the kind of letter that came before changed the odds of what would follow. Language contained a pattern that mathematics could describe, even when the calculation knew nothing about the poem’s meaning.

Markov was investigating probability. He was not trying to build a talking machine. But his experiment gives us a place to begin: what comes next depends on what came before.

At Bell Labs, Claude Shannon was working on how to transmit messages reliably. His theory separated the engineering problem of carrying a message from the meaning of the message itself.

He also showed what happened when you generated text using progressively richer patterns from English. Random letters looked like noise. Letter combinations began to resemble words. Word pairs produced stretches that almost sounded as though someone had something to say.

These were samples assembled from statistical patterns, some by consulting books by hand. Yet they exposed a possibility: a process could produce recognisable features of language without anyone writing its sentences in advance.

It could also produce nonsense. That distinction would become central.

Noam Chomsky offered a sentence that makes the problem easy to hear:

Colorless green ideas sleep furiously.

Now reverse it:

Furiously sleep ideas green colorless.

The first has the structure of an English sentence, despite its strange meaning. The second has the same words, but loses that structure.

Chomsky challenged the idea that the statistical approaches of the time could adequately explain grammar. A theory of language had to account for sentences people had never encountered, including ones that made no sense.

The question was becoming harder. Could a machine learn relationships between words well enough to handle something new?

For researchers trying to make computers recognise speech, this was an everyday engineering problem. Sound is ambiguous. The words around a sound help us decide what we heard.

At IBM, Frederick Jelinek’s group used probabilities learned from text to help choose between possible word sequences. At Carnegie Mellon, James and Janet Baker developed a speech recogniser using hidden Markov models. Statistical methods were finding practical uses while arguments about language continued.

In 1990, IBM researchers published a translation system that learned from paired French and English records of Canada’s Parliament. They were applying statistical methods developed in speech recognition to a different job.

The approach was limited, but the connection mattered. Mathematics used to recognise spoken words could also help translate written ones. Language models were becoming working parts of useful systems.

A table of word sequences has a problem: people can always say something that is missing from the table.

Neural language models offered a way to generalise. In work published in 2003, Yoshua Bengio and colleagues trained a model to learn numerical representations of words along with how to predict them. Words used in similar ways could share what the model learned.

That meant an unfamiliar combination could benefit from familiar ones. The model did not need a separate entry for every possible sentence.

This was a consequential change in how language was represented. The system was still predicting what came next, but it was learning relationships that a simple tally could not express.

A useful clue may be several words away. In “the keys to the cabinet are missing,” the word “keys” helps determine “are,” even though “cabinet” is closer.

The Transformer, introduced in 2017, used attention to learn relationships between positions in a sequence. Its design also allowed much more of the training work to happen in parallel, making better use of computing hardware. The original paper demonstrated it on translation.

Later language models combined this architecture with much larger training runs. They learned to continue text across a widening range of subjects and tasks. By 2020, GPT-3 demonstrated that examples written into a prompt could guide a single model through different tasks without retraining it for each one.

Prediction had become a way to train a remarkably versatile system. Getting that system to do what a person actually wanted was another problem.

A model that can continue a document is not automatically a good assistant. It may continue your question instead of answering it, ignore your intention or give a fluent answer that is wrong.

Instruction training and human feedback helped close that gap. Researchers showed models examples of useful responses and used people’s preferences to improve their behaviour. InstructGPT, published in 2022, demonstrated how this could make a model better at following a user’s request.

That changed the experience of using the technology. A person could describe a task in ordinary language, examine the response and ask for a revision.

The mathematics had reached an interface people already knew how to use: a conversation.