Type a question into an AI assistant. It responds in seconds — fluent, confident, often correct. You close the tab.
What just happened is stranger than it looks. My last post asked whether to trust it. This one goes a level deeper: what is it actually doing? Because once you see the machinery, a few genuinely surprising things fall out — not just about AI, but about language itself.
From Words to Numbers
Before a model can do anything with your message, it has to convert it into a form it can process. It doesn't read words. It reads tokens — fragments that sit somewhere between letters and full words — and converts each one into an integer.
"Indistinguishable" might become four tokens. "The" is one. A seven-word sentence might yield nine tokens; a technical term you coined last week might be broken into pieces the model has seen before in other combinations.
Fig. 1 — Tokenisation: "I love programming. It's awesome."
Seven words become seven token IDs — the only form the model actually sees
Every token gets a number. That sequence of numbers — nothing else — is what enters the model. But numbers alone carry no meaning. Which brings us to the part that actually matters.
Meaning as a location in space
Each token ID is converted into a vector — a list of thousands of numbers representing its meaning. Think of it as coordinates in a vast space where similar concepts end up near each other.
Nobody told the model that "Python" (the language) and "JavaScript" are related. No one annotated millions of documents to mark those two words as neighbours. The model found the proximity itself — because these words appear in similar contexts, surrounded by similar other words, across millions of documents.
That's all it took.
Fig. 2 — Embedding Space: related concepts cluster together
Cluster A · Royalty
Cluster B · Programming
Cluster C · Data
Nobody told the model these words were related — it learned proximity purely from patterns in text
Note
The famous demonstration: take the vector for king, subtract man, add woman — and you land near queen. First demonstrated in simpler models, but the principle extends to modern embeddings: relationships like gender can be learned purely from text patterns, with no explicit instruction (Mikolov et al., 2013).
The Spotlight: How Context Resolves Meaning
These vectors flow into the transformer — the neural network at the heart of every modern language model. The key mechanism inside is called attention, and it's what separates these models from simpler pattern-matchers.
Consider: "The cat sat on the mat because it was tired."
What does it refer to? The cat. Not the mat. You resolved that instantly. The model does it through attention — when processing the word it, it assigns high weight to cat and low weight to mat, even though mat is closer in the sentence.
Fig. 3 — Attention: processing the word "it" — which tokens does the model focus on?
This calculation happens through multiple "attention heads" running in parallel within each layer — each capturing a different type of relationship: grammatical, semantic, referential. Those layers stack sequentially. Each pass builds a more abstract understanding of the whole.
By the end, the model holds a rich representation of what your words mean in context — relative to each other, and to everything it absorbed during training.
One Hundred Thousand Guesses at Once
After processing your input, the model doesn't decide what to say. It produces a probability score for every single token in its vocabulary — up to 128,000 options — and one is selected.
Fig. 4 — Probability distribution: "What is Python? Python ___"
Every token in the vocabulary gets a score. One is selected. Then the loop begins again.
One token is chosen. Then the whole process runs again with that token appended to the input. Then again. And again, until the response is complete.
Fig. 5 — The generation loop: one token at a time
The model has no idea what token 10 will be when it's generating token 3
This is why the model genuinely doesn't know what it will say in advance. There's no script waiting to come out. Token 50 hasn't been determined when token 3 is being generated. Each word is decided only when it's that word's turn, based on everything that came before.
It's also why my previous post's central point holds up mechanically: there's no reasoning happening here in the human sense. There's a probability distribution being sampled. That distribution is shaped by an enormous amount of learned pattern — but it doesn't know anything. It assigns likelihood.
So How Does It Actually Answer Questions?
Token-by-token generation explains the how. It doesn't explain the why — why the output tends to be relevant, coherent, and shaped like an answer rather than a random continuation.
Three things work together.
The weights store knowledge. The billions of parameters inside the model are a compressed memory of patterns absorbed during training — including countless examples of questions followed by answers. When your question arrives, relevant patterns activate.
The whole question steers every prediction. Because your full question sits in the context window throughout generation, the probability distribution for every new token is shaped by everything you asked. The question pulls likely tokens toward answer-shaped outputs.
Instruction tuning teaches the answering behaviour. A model trained only on raw internet text would just continue text — responding to a question with more questions, or unrelated prose. What makes modern AI assistants actually answer is a separate training phase where the model is fine-tuned on examples of questions paired with good responses.
None of this requires understanding. It requires enough exposure to the pattern of question-then-answer that the model reliably reproduces it.
The Rules Nobody Wrote Down
Here's where things get philosophically interesting.
Nobody gave these models a grammar book. No linguist annotated the training data. No rules were programmed in. And yet the models emerged from training with fluent command of syntax, tense, reference, metaphor, and register — across dozens of languages simultaneously.
Fig. 6 — How language rules emerge without being taught
This mirrors how children actually acquire language — not by learning rules first and then applying them, but by absorbing enough examples that the patterns become instinctive, long before they could explain what a subjunctive clause is or why "I went" sounds right and "I goed" doesn't. Linguists call this implicit learning. LLMs do the same thing, at a scale no human could experience in a lifetime.
This raises a question that linguists haven't fully settled. Noam Chomsky proposed that humans have an innate "language organ" — a biological blueprint for grammar that explains why children acquire it so rapidly. But if the deep rules of language can be learned purely by exposure, without a blueprint — which LLMs appear to demonstrate — what does that imply?
Either the rules aren't as innately human as we thought. Or they're so deeply embedded in the structure of reality that any system exposed to enough human communication will find them, regardless of architecture.
The more interesting possibility: language rules may be an inevitable consequence of any sufficiently complex communication system trying to reliably carry meaning. Not divine inspiration, not biology — something closer to physics. Given enough minds trying to share enough meaning over enough time, certain structures become the only viable solutions. They emerge the way rivers find the sea: not by design, but because the terrain made certain paths inevitable.
What If Two AIs Talked Without Us?
If language models needed to communicate with each other — no humans in the middle — they almost certainly wouldn't use human language at all.
Human language is fundamentally a compression system. You take a rich, multidimensional thought, squeeze it into a linear sequence of words, transmit it, and hope the other person reconstructs something close to the original. Every step loses something.
LLMs don't think in words. They think in the vectors described above — high-dimensional mathematical objects where thousands of aspects of meaning are encoded simultaneously. The most efficient LLM-to-LLM communication would bypass language entirely and transmit those vectors directly. No tokenisation. No grammar. No ambiguity.
Fig. 7 — Two communication channels compared
The research confirms it. When AI systems are given coordination tasks and allowed to develop their own protocols, what emerges tends not to be recognisable language — it's efficient, task-specific signalling that humans struggle to interpret. Not because it's hidden, but because it operates in a space we have no natural way to perceive.
This exposes something worth sitting with: human language isn't the pinnacle of communication. It's a good enough solution to a very specific problem — getting meaning from one biological brain to another using only sound or marks on a surface.
What the Mechanism Reveals
The machinery described here produces the outputs my previous post warned about — the confident, convincing responses that can pass as understanding without containing any.
But understanding the mechanism also changes how you relate to it. When an LLM sounds authoritative and turns out to be wrong, it's not being deceptive. It's producing text that pattern-matches to what a confident, correct-sounding response looks like. The probability distribution has no concept of truth. It only knows plausibility.
The implication isn't that these tools are useless. It's that the skill isn't just knowing how to use them. It's knowing what you're looking at when you do.
Tokens in, meaning vectors, attention connecting context, probabilities over a hundred thousand options, one selection at a time. It's not magic. It's mechanism.
And the mechanism, once you see it, is strange enough to change how you read every AI response that follows.