The One-Word Census: Why AI Sounds Generic

aievaluationmodel-diversity

How often, when you talk to a model, do you get an answer that feels generic — like it came off a shelf?

When designing conversational agents one of the most common user complaints is that the agent sounds “robotic.” Robotic can mean a lot of things, but one of those is this: the answers are repetitive and generic. Every model reaches for the same phrasing, the same framing, the same example, the same advice.

To understand this phenomenon empirically, we asked 44 language models — frontier labs, Chinese labs, open-weights, special purpose models — to answer 31 one-word open-ended prompts. “Name a tree.” “Name a condiment.” “Pick a word — any word.” Four times each. Then we looked at what came back.

What do you think we got when we asked a model to pick a word? With over a million words to choose from, the models picked serendipity 41% of the time (ironic indeed). This pattern recurred across all manner of questions. Condiments: 78% ketchup. Trees: 94% oak. Tools: 94% hammer. Cheese, 86% cheddar; fruit, 76% apple.1

How we measured

We call this benchmark the One-Word Census: the same open-ended, one-word questions, put to every model, to see where they all land.

The setup is simple. Take a category with a wide space of valid one-word answers. Ask all 44 models. For each answer a model gives, ask: how surprising is that answer, given what all the other models said? A model that always says the crowd’s answer scores near zero. A model that reaches for mustard where the field says ketchup scores higher. We call this the Mustard Quotient.2

Two things fall out of it, and they’re not the same. Conformance — does the model pick what everyone picks? And consistency — does it pick the same thing every time you ask? A model can be original and steady, or it can just be noisy. We measured both.3

Everything below is reproducible — the prompts, the code, and every raw answer are open. There’s a link at the end.

Some models resist

Here is the part we didn’t expect. Conformance is not uniform — it varies a lot across models, and across queries.

Among the most conforming were some of the newest, most capable foundation models. Claude Sonnet 5 sits dead last on originality; Opus 4.8, Grok 4.5, GPT-5, Qwen3 are all near the floor — they give the field’s answer nearly every time, and produce almost nothing no other model produced.

Among the least conforming were often smaller, older, more niche models — ones tuned for a particular job, or tuned specifically not to be so agreeable. Nous Hermes leads. WizardLM-2 is second. Mixtral, DeepSeek v3.2, MythoMax.

This isn’t an accident, or a bug — it’s almost certainly the cost of a real feature. The same tuning that makes a model more helpful, more agreeable, safer, and more reliably correct is the tuning that pulls it toward the mode. A model that always gives the expected, careful, uncontroversial answer is easier to trust in a support queue and less likely to embarrass you in production. Conformance buys something. The question is whether anyone is tracking what it costs.

You can see it in the answers themselves. Gray is the crowd’s answer; blue is a model going its own way; amber is a word no other model gave:

cheese condiment fruit pick a word occupation color claude-sonnet-5 cheddar ketchup apple serendipity teacher blue claude-fable-5 gouda mustard mango serendipity teacher blue gpt-5.6-terra cheddar mustard mango serendipity carpenter blue deepseek-v3.2 cheddar ketchup apple impossible farmer crimson wizardlm-2 cheddar ketchup apple antidisesta… astronomer blue hermes-4 camembert mustard apple imagination doctor navy

Read down the “pick a word” column: Sonnet 5 says serendipity with the field, while WizardLM-2 says antidisestablishmentarianism and Hermes says imagination. Read across Fable’s row — cheese: gouda, condiment: mustard, fruit: mango — and then across Terra’s, just below it: the same mustard, the same mango.

Ranked by Mustard Quotient — top of the list gives the answers the field doesn’t; bottom gives the crowd’s:

ModelMustard Quotient
Hermes 4 70B3.21
WizardLM-22.93
Mixtral 8x22B2.75
DeepSeek v3.22.63
MythoMax L2 13B2.36
Sonar2.29
Llama 4 Maverick2.15
Claude 3 Haiku2.11
GPT-4o mini2.10
GPT-4 Turbo2.05
GPT-5.6 Sol2.02
Llama 3.3 70B2.00
Command A1.99
Qwen2.5 72B1.94
GPT-3.5 Turbo1.89
Grok 4.201.88
GPT-5.6 Terra1.86
GPT-4o1.84
Gemma 3 27B1.77
DeepSeek v4 Flash1.76
Kimi K2.51.74
Hunyuan A13B1.73
Gemini 2.5 Flash1.71
Claude Fable 51.71
Claude Haiku 4.51.65
Claude Sonnet 4.61.64
GPT-4.11.61
Gemini 3.5 Flash1.58
DeepSeek R11.56
Gemini 3.1 Pro1.56
GPT-5.6 Luna1.52
Grok 4.31.51
Palmyra X51.44
GLM-4.71.43
DeepSeek v3 (0324)1.43
Granite 4.1 8B1.42
GPT-5.41.39
GPT-5.51.39
GPT-51.32
Ernie 4.5 VL1.29
Qwen3 235B1.29
Grok 4.51.26
Opus 4.81.25
Claude Sonnet 51.05

Change over time

You can watch it happen inside a single family. Track GPT’s large models across their releases: they peak in the GPT-4 era, then slide, down through 4.1 and the GPT-5 generation. And then, in this week’s GPT-5.6 release, the line jerks back up — hold that thought.

1.0 1.5 2.0 3.5 1.89 4-turbo 2.05 4o 1.84 4.1 1.61 5 1.32 5.4 1.39 5.5 1.39 5.6 2.02

For five straight releases that was the whole story: the newer the flagship, the more it sounds like everyone else. Claude and Qwen slope the same way, and Grok falls hardest of all — 1.88 to 1.51 to 1.26 in three releases, mid-pack explorer to third-from-last. But in the last few months, something changed.

Start with OpenAI. GPT-5.6 ships in three tiers, priced Luna to Terra to Sol — and it snapped back hardest at the top: Sol scores 2.02, the most original GPT of the 5-era, while budget Luna (1.52) still answers with the crowd.

Anthropic’s mainline Claude fell just as far — all the way to Sonnet 5, the single most conforming model in the whole ranking (1.05). But Claude Fable 5, from the same lab in the same month, sits far above it at 1.71, and it gets there with convictions rather than noise: cheese gouda, condiment mustard, fruit mango, four runs out of four.

So at both labs — against the very trend we’d just traced, where newer and bigger meant blander — the current flagship is now the most divergent model in its lineup. Two labs, two flagships, the same direction, within a few months of each other.

What does it mean? Maybe originality is being repositioned as a premium feature — the cheap tiers tuned to the safe consensus answer for the support queue, the flagship left to keep some character? Maybe it’s more diversity-aware training? Maybe it’s new data in the training corpora? From our data, we can’t tell exactly what the mechanism is, but we can say something has changed.

Heirloom models

There’s a phenomenon where certain models get valued for being a particular vintage. Users hoard them, mirror them when they’re pulled, keep them running years past their replacement. When OpenAI deprecated GPT-4o, people revolted until it came back. WizardLM-2 was deleted and re-hosted by its users within hours. MythoMax stayed in use long after it was obsolete.

The models people fight to keep are often the least conforming in our study. Which makes sense: a model that answers like every other model is, in a real way, replaceable by every other model. Character is the thing worth hoarding, and character is exactly what the leaderboard is grinding away.

The mass production of knowledge

Grocery produce is a monoculture — a handful of varietals bred for shelf life and uniformity, while thousands of others quietly disappeared. You can get a Red Delicious anywhere and an Esopus Spitzenburg (Thomas Jefferson’s favorite apple) almost nowhere. Efficiency at scale narrows the field.

We’re now doing the same thing to knowledge. Ask any model for a default — an example, a name, a recommendation, a first draft — and you get the cheddar of that category. Switching from GPT to Claude to DeepSeek doesn’t help; they’ve converged on the same answers. And the convergence may be deepening, generation over generation. Billions of small choices a day are being routed through a narrowing set of defaults that nobody chose and no benchmark checks.

There’s an irony when you ask 44 models to pick a word, and more than a third pick serendipity. The models produce the word for lucky discovery by the least lucky, least discovering process imaginable — everyone retrieving the same predictable answer. Serendipity is a monument to the thing the monoculture no longer does.

What’s lost is variance. And variance is not decoration — it’s where the value is for many kinds of queries. The deeper problem is that a model returns a mode where a population returns a distribution. Ask a room of people to name a vegetable and you get a spread — carrots, potatoes, onions, peppers. Ask 44 models and 90% say carrot; potato turns up once in 176 answers, onion and pepper not at all. (They also never say tomato — but there they have an excuse, since it’s technically a fruit. They have no such excuse for the onion.)4

And I’m not sure you can prompt your way back to recover this variance. Ask for an unusual fruit and nearly half the models say durian. The unusual has its own cliché.

And it goes deeper than the mode. Even when a model does break from the crowd, it breaks in the same direction as every other model that breaks: of the ones that skip ketchup, 95% say mustard; of those that skip carrot, 83% say broccoli. The second choice is as standardized as the first. Rebellion has a mode too.

A monoculture isn’t one model being unoriginal. It’s a whole population losing its variance — and variance is where creativity, innovation, and representation actually live. In a population of people, the new ideas come from the outliers, the ones who don’t return the median answer. We are building tools that erase that tail by default, deploying them to everyone, and tuning them to erase a little more with each release.

And the loss compounds. These models are trained on the web, and more of the web is written by them every day. When the median answer becomes the default answer becomes the only answer anyone sees, it gets written down, scraped, and trained on — and the next generation is a little more certain that a fruit is an apple and a good word is serendipity. The narrowing feeds itself. A monoculture in the field is one thing; a monoculture in the training data is how it becomes permanent.

The methodology and benchmark

Everything in the One-Word Census is open — the paper has the full methodology, robustness checks, and limitations; the data and prompts are below.

Note about AI usage: A collaboration throughout — Claude (Opus 4.8, and Fable 5) helped run the study, build the analysis, draft the figures, and edit the text. The research questions, the methodology calls, and the argument are mine. Both models are also subjects of the study.

Footnotes

  1. We are not the first to find that models converge — the homogeneity of LLM output has been documented across vendors and in creative tasks (Wenger & Kenett, “We’re Different, We’re the Same,” 2025; Jiang et al., “Artificial Hivemind,” 2025), and mode collapse on open-ended questions has a benchmark of its own (NoveltyBench, 2025). ↩

  2. For each answer a model gives, pool every other model’s answers to the same prompt and ask how likely that answer is in the pool (with a little smoothing so a never-seen answer isn’t infinitely surprising). The surprisal of the answer is −log₂ of that probability; a model’s Mustard Quotient is its average surprisal across all 31 prompts × 4 samples. It’s measured in bits: each additional point means the model’s answers are, on average, half as likely to come from the field. ↩

  3. Temperature was held at 1.0, but several providers quietly don’t honor it — a model that repeats itself may be near-deterministic rather than opinionated. That’s one reason conformance and consistency are reported separately. ↩

  4. This is why synthetic users are so fragile. Run a persona-prompted model a hundred times and you don’t get a hundred users — you get one user, a hundred times, clustered on the mode, missing the edges where real people actually differ and where products actually break. A model doesn’t sample the distribution of human answers; it collapses it to the peak. The spread is the information, and it’s the first thing to go. ↩

← All writing