Knowledge Distillation Is Quietly Deleting Your Brand From Small Models
Three under-discussed links between distillation, test-time compute, and perplexity — and why the model answering your buyers never read your best page.
The model deciding whether to recommend you is usually not the frontier flagship. It's a distilled student — an 8B model serving autocomplete-priced inference inside a search product, a support bot, or an agent stack. Distillation is lossy compression. Long-tail brand knowledge is exactly what gets lost. If your GEO strategy assumes every answer comes from the biggest model, you're optimizing for a jury that rarely shows up.
Distillation keeps distributions, not documents
A student model never reads your content. It learns to imitate the teacher's output distribution over sampled prompts. That has a brutal consequence for brands: weak, low-probability associations get smoothed away. If the teacher mentions your brand in 3% of category prompts, the student rounds you to zero. Your competitor at 30% survives compression; you don't.
Hidden connection #1: your visibility in small models is not a function of content quality. It's a function of how consistently the teacher already associates your entity with the category. Distillation amplifies consensus and deletes ambiguity. GEO for small models is therefore played entirely upstream, in the frontier models the students are distilled from.
Test-time compute is your appeal process
Thinking modes change the mechanics of an answer. A System 1 response emits the prior: whoever dominates the weights wins. A System 2 response decomposes the query, fires sub-queries, retrieves, and cross-checks claims mid-chain. That's a second jury — and it has subpoena power over the live web.
Hidden connection #2: distillation deletes you from the prior; test-time compute is the only mechanism that can put you back. But reasoning chains audit claims. If your pricing page, docs, and third-party mentions disagree with each other, the chain flags the inconsistency and drops you mid-reasoning — a failure mode invisible in the final answer. Verifiable, mutually consistent facts aren't hygiene; they're survival criteria for chain-of-thought retrieval.
Perplexity is the invisible citation filter
At the token level, models preferentially reproduce low-perplexity spans — phrasing that is cheap to predict. If your brand facts exist in one canonical formulation (name + category + differentiator, repeated verbatim across your site, docs, and profiles), emitting them costs the model almost nothing. If every page describes you differently, token-level surprise rises and the model paraphrases around you — often keeping your idea and dropping your attribution.
Hidden connection #3: the same canonical consistency that survives distillation also lowers the perplexity of your brand facts at inference. One fix drives two mechanisms. This is why entity-consistent boilerplate outperforms creative variation in GEO, even though it feels wrong to every copywriter.
Measure the distillation gap
You can't audit anyone's distillation pipeline. You can measure its output. Run the same category prompts against frontier and small/fast tiers and diff the answers. Present in the flagship but absent in the mini tier? That's the distillation gap — and it's where most of your buyers' queries actually land.
LLM Search Console does this systematically: it tracks your prompts across ChatGPT, Gemini, Perplexity, and Claude, scores brand mentions, citations, and share of voice per model, and shows you where you exist in one engine and vanish in another. That per-model delta is the single most actionable GEO signal you can get — it tells you whether your problem is the prior, the retrieval layer, or the phrasing.
Quick wins for GEO
Canonicalize your entity sentence. One formulation of name + category + differentiator, verbatim, everywhere. Low perplexity, distillation-resistant.
Reconcile your facts. Pricing, feature claims, and founding data must agree across your site, docs, and third-party profiles — reasoning chains cross-check them.
Test small tiers, not just flagships. Ask the mini/flash models your category questions. That's the inference actually serving volume.
Win co-occurrence, not just content. Get your brand named next to the category in sources frontier models trust; students inherit what teachers repeat.
Track the deltas continuously. Set up prompt tracking in LLM Search Console and watch per-model share of voice weekly. Distillation cycles ship quarterly; your visibility can drop without any change on your side.
Rankings measured pages. AI answers measure entities. Make yours cheap to predict, cheap to verify, and impossible to compress away.