A Language Model Is Two Files

This section explains what a language model is made of when it is stored on a computer, and what it does each time it runs.

The Parameter File

A language model is a computer program that predicts text. When a language model is stored on a computer, it can be described as just two files. One example is Llama 2 70B, a model that Meta released for anyone to download and run. Parameters are the numbers a language model uses to calculate what text comes next. The larger of the two files holds the parameters. Llama 2 70B has 70 billion parameters. Each parameter is stored as a number that takes up 2 bytes, so the parameter file for this model is 140 GB. A person who opened that file would see a long list of numbers and would not find a single sentence written out in it. A parameter file is not an example of a saved document, because a saved document holds its sentences and gives them back exactly. The parameter file holds no sentences, so its numbers are useful only when a program reads them and does arithmetic with them.

Where the Parameters Come From

The parameters are set by training, a process in which a computer adjusts the numbers while it processes a very large amount of text. For Llama 2 70B, the training data was about 10 TB of text taken from the internet. One useful way to think about the result is as lossy compression of the text that was used in training. Lossy compression makes a file smaller by removing detail that cannot be recovered, like the JPEG photos from last week. The 140 GB parameter file is far smaller than the 10 TB of text, so it cannot hold an exact copy of that text. This is a comparison and not an exact description of what training does. A JPEG still stores a picture, while the parameter file stores only numbers that help predict text.

The Runtime

The second, much smaller file is the runtime. A runtime is the program that reads the parameter file and does the arithmetic that turns those numbers into text. Andrej Karpathy's 2023 talk "Intro to Large Language Models" describes a runtime for Llama 2 70B that takes about 500 lines of code in the C programming language. Another example is llama.cpp, a runtime written in C and C++ that needs no other software libraries to work. A runtime with no parameter file has nothing to run, because the arithmetic needs the numbers that the parameter file stores. A parameter file with no runtime sits on a drive and produces nothing, because no program is reading its numbers. When both files are on the same computer, they form a complete package, and the model can run without any connection to the internet. Mr. Mugg's computer at home runs open models this way, as two files stored on a drive. MuggsOfPrompts and MuggsOfVibes, the tools used in this class, work differently. They send each prompt over the internet to a model that runs on Mistral's servers.

Prediction, One Token at a Time

A model reads and produces text as tokens, which are words or parts of words that a computer represents as numbers. Prediction is the process of producing a likely next token from the text so far. The runtime carries out prediction in a loop of four steps, as Figure 1 shows. First, it reads all the tokens so far, starting with the prompt. Next, it uses the parameters to score every possible next token by calculating how likely each one is. Then it picks one token. Finally, it adds that token to the end of the text and starts the loop again with a list that is one token longer. For example, given the words "cat sat on a," a model might predict "mat" as the next word. The model repeats the loop one token at a time until the answer is complete. Prediction is not the same as a lookup, because a lookup hands back something that was stored, while a prediction produces something new. For this reason, a model can generate a sentence that does not appear anywhere in its training text. That sentence may be correct, or it may contain details that are false.

Summary

A language model on a computer is two files: a large parameter file that stores numbers and a small runtime that does the arithmetic with them. Each time the model runs, it makes a prediction of the next token, adds that token to the text, and repeats. Training on a very large amount of text set the parameters, and lossy compression is one useful comparison for the result, though not an exact description.

Sources: Karpathy, "[1hr Talk] Intro to Large Language Models" (2023); Meta, Llama 2 Model Card (2023); Touvron et al., "Llama 2: Open Foundation and Fine-Tuned Chat Models" (2023); ggml-org, llama.cpp README (2026); Hugging Face, LLM Course, "Tokenizers" (n.d.); classroom data (2026).