Where the Numbers Came From

This reading explains how training set the numbers in a parameter file, and why a detail from training can come back wrong.

Comparing Two Sizes

Training is the process that sets a model's parameters. During training, a computer processes a very large amount of text and adjusts the parameters so that the model predicts the next token more accurately. Meta's model card states that Llama 2 was trained on 2 trillion tokens of data from publicly available sources. In his 2023 talk, Andrej Karpathy describes the training text for Llama 2 70B as about 10 TB. The parameter file that came out of training is 140 GB. These two sizes can be compared directly, because 10 TB is about 10,000 GB. The text is about 70 times as large as the parameter file. A file that size cannot hold an exact copy of the text, so some detail from the text was not kept. This is why lossy compression is one useful way to think about training, even though training is not the same process as saving a JPEG. A token is the unit a model reads and writes. A token is often a whole word, but it can be a part of a word or a punctuation mark, so the 2 trillion token count depends on the tokenizer that Meta used. The 10 TB figure is the size of the raw text on disk, and the 140 GB figure is the size of the parameter file that training produced from that text. Neither number is the training recipe. Training is a long sequence of small adjustments to the numbers, each one made because a prediction landed a little closer or a little further from the next token in the training text.

Why Details Come Back Wrong

When a model generates text, it predicts each token from patterns stored in its parameters. Some details from the training text are stored closely, and others are not. In the talk, a model generated a page that looked like a book listing, with a title, an author and an ISBN number. Karpathy explains that the ISBN number was almost certainly made up. The model produced a number of about the right length in the place where an ISBN usually appears, but the digits themselves were a prediction and not a stored record. A detail that appeared only once in 10 TB of text is a very small part of what the model was trained on. A person reading the output cannot tell from the text alone which details were stored closely and which were not. This is why a model's answer should be treated as a draft that needs a check against a source. A fact that is likely stored closely, such as a widely-quoted sentence from a very common book, often comes back the same way every time. A fact that is rarely written down, such as a specific ISBN or the date a small company was founded, is more likely to come back as a plausible-looking guess. The reader has no way to tell the two apart from the text alone, which makes checking the first step and not the last.

Sources: Karpathy, "[1hr Talk] Intro to Large Language Models" (2023); Meta, Llama 2 Model Card (2023).