How a translation model chooses which input words to look at before it writes each output word. Running example, the start of our sentence: Unha tía miña, que todo o solucionaba con ditos e refráns, dicía algo así como «un é escravo do que di e dono do que cala». It continues the neural translation page (translation without attention).
What you will learn on this page
Without attention, the decoder gets one summary of the whole sentence, and a word read long ago has faded from it. → sections 1, 4
With attention, before each English word the decoder builds a new mix of all the encoder's vectors: query · each key → scores → softmax → weights → weighted sum = the mix. → section 2
The query's numbers decide which key is read, and so which word is written; each word needs its own query. → section 3
The evidence: two models alike except for attention. With 8 dimensions, only the one with attention translates our long sentence; with 64, both do (section 5). → sections 4, 5
A grid row is one word's weights. A head is one run of query, scores, weights and mix with its own learned tables: one grid. A layer runs several heads side by side; a real model stacks layers. The weights show where the model looks, not whether its output is right. → section 6
Self-attention: every token (a word or part of a word) looks at every token. Heads specialize during training. → sections 7, 8
1Without attention, computed by hand
Review
From the translation page: the encoder reads the Galician words into one vector, the summary; the decoder writes the English words one at a time. From the skip-gram lesson: a vector is a list of values, one per dimension; the dot product multiplies two vectors dimension by dimension and adds up the results.
Review
The example translations: the 16 phrases, translated by people, that the translation page's model was trained on.
a combination it never saw: a matter of the training data, not of the summary
translation page, section 1 part D
the same 15, trained longer
an aunt of mine
from round 4217 on
sections 1 and 2 here
nothing: made-up values
a friend of mine; with attention, an aunt of mine
made up to fail the way a model without attention fails on our long sentence, where 29 more words and marks follow tía
sections 4 and 5 here
5,000 template sentences; ours held out
A friend of mine, …; with attention, An aunt of mine, …
same data, same number of updates, same size: the difference is attention's
Step 5: how training on these would produce such values.
A made-up model, small enough to compute by hand, on unha tía miña: vectors of three dimensions, eight English words to choose from, every number shown. Its values are made up so that it fails like a model without attention on a long sentence: it writes “a friend of mine”. Section 2 adds attention: “an aunt of mine”.
2With attention, computed by hand
Section 1 wrote “a friend of mine”: the summary held 0.25 of tía, and “an” needs more than 0.35. The decoder state has eleven dimensions (section 1, step 3): the first three, one per Galician word, hold what came from the encoder; the other eight record the English words written so far. Attention changes only the first three: before each English word, the decoder builds a new vector from all three encoder vectors, the mix, and puts it there in place of the summary.
without attention (section 1)
after unhaafter tíaafter miña = the summary→the state's first three dimensions (from the encoder)
with attention (this section)
after unhaafter tíaafter miña→dot product with the query: three scores→softmax: three weights→weighted sum: the mix→the state's first three dimensions (from the encoder)
3Query and key: what the query's numbers mean
Section 2's query was given ready-made. Here you set it yourself and follow the chain query → scores → weights → mix → word, to see where it works and where it fails.
the decoder is about to write word
Galician word vectorencoder vectorEnglish word vectorquerythe summarythe mix: what enters the decoder state
Orange arrow: the query's direction (drawn short). Each dashed line drops a key onto it; where it lands is the key's score.Drag to turn; double-click to reset.
the same query at every word of the phrase
One row per English word: the correct words before it are written, and the state's first three dimensions, the part that comes from the encoder (section 1, step 3), hold this query's mix.
4The same steps in two trained models
Two new small models translate our whole sentence. Same data, same training; one difference: without attention, the decoder gets the summary for every English word; with attention, a new mix of all the encoder's vectors, as in section 2. Their vectors have 8 dimensions, all values learned in training.
one square = one dimension of a vector; blue positive, pink negative (key in the strip below). Every number stays between −1 and +1.
−1+1
5The two models, word by word
The two models of section 4, word by word: on our sentence, its three parts (the noun phrase, the clause, the saying) and five test sentences, at three vector sizes. Same data, same number of updates, same number of parameters (learned values). Both train on 5,000 sentences built from templates around ours (other relatives, verbs, sayings). Another 500, and our sentence, are kept apart as test data: never trained on, never used for choices, only for measuring at the end. Their words all appear in training, in other combinations.
Vector size
Gold translation
or step through each model with its own buttons below
−1+1one square = one dimension of a vector, between −1 and +1. Under each model: its five most probable next words; orange = the word it writes; ✓ = the gold word (while the words so far match the gold translation). With attention: line thickness = weight.
Without attention
→
With attention
→
Where the weights come from
Exactly right, out of 500 test sentences (not in the training data)
Words in red are not in the gold translation; struck-through words are missing. With 64 dimensions, one vector holds this toy data perfectly; real models of 2014 lost details in long sentences (Cho et al. 2014; Bahdanau et al. 2015). Attention solved this without a bigger vector.
6A real model on our sentence
Sections 4 and 5 compared two small models we trained. Now one real translation model, opus-mt-gl-en, trained by others: its weights, drawn as grids, first on our phrase, then on the whole sentence. One question to carry through: just before it writes “aunt”, how much weight does it give tía, and what does that tell us where the translation goes wrong?
The model whose grids we read
one head: section 2's computation, with three learned tables
one layer: 8 heads side by side
the decoder: 6 layers, one after the other
Reading the grids
What to take from this
The whole sentence
Tokens: what the model reads and writes
tía↓tía
solucionaba↓solucionaba
ditos↓ditos
refráns↓refráns
,↓,
(the end of the sentence)↓‹end›
A token is one entry of the model's fixed list of 21,305 entries: a whole word, part of a word, a punctuation mark or ‹end›. Common words are one token; rarer ones are split. In the phrase above every word was one token; from here on, each row and each column of a grid is one token. In the menus, average means the grids averaged cell by cell: a layer's 8 heads, or all 48.
Try:
The weights show where the model looks, not whether what it writes is right
Three places where the translation goes wrong:
Click one of the three places above: its explanation appears here.
The same picture as section 5, now for a real model: before the English token in orange, lines to the Galician tokens it puts weight on, as thick as the weight (lines under 0.01 left out); click an English token
The same weights for all English tokens at once: one row per English token, one column per Galician token
Layer by layer: two cells of the grid above, each averaged over the 8 heads of the layer
7Self-attention, and many heads at once
In the encoder, every Galician token looks at every Galician token, itself included: self-attention. It takes the place of the chain of sections 1 to 5.
Above, each vector waits for the one before it; below, none waits. Orange: the links of miña, each with a weight. To keep the order, each token's vector first gets a vector for its position (1st, 2nd, …) added.
How one layer combines its 8 heads, for one token
head 1: a mix for miñahead 2: a mix for miña…head 8: a mix for miña→the 8 mixes side by side as one long vector, × one learned table→+ the vector of miña→one more learned step→the vector of miña in the next layer
“+ the vector of miña”: added to what miña had, so nothing is lost. Decoder layers do the same.
The 8 heads of one layer: one square grid each, a row and a column per token; rows look, columns are looked at
Diagonal (top left to bottom right): each token looks at itself. Just below it: at the token before (layer 6, head 7); just above: at the token after (layer 3, head 4).
Pick a head, then click a Galician token.
8How heads specialize
Every head starts random, and training nudges all of them at once (translation page, section 1). Some heads end up with a job we can recognize in their grids. To see this we use a model studied for exactly this question.
A different real model: BERT (bert-base-uncased)
English text, some words hidden→encoder as in section 7: 12 layers × 12 heads→the hidden words
No decoder and no translation: BERT is trained to fill in hidden words. It adds two tokens to every text: [CLS] at the start and [SEP] at the end. [SEP] plays the part of ‹end› in section 6: a head with nothing to do often puts its weight there.
Six heads, one sentence: My aunt solved every problem with a story.
Click a head: 144 in all, these six here
The orange row as lines, as in section 5; click a token
“it”, in two sentences that differ in one word
Ask Claude
Ask about anything on this page: what a panel shows, why a vector moved, what a number means. Claude sees the page's current state.