"You are a doctor" isn't enough — what an expert prompt actually is
2026-04-28 · Shin Chi-hoon
An empty mask with the title 'doctor' on it carries no knowledge in itself. Real expertise lives in the structure of facts — patient, symptom, diagnosis, treatment, outcome — that connects underneath.
Anyone who uses ChatGPT or Claude learns one technique first. "You are a doctor." "You are a lawyer." "You are a senior software engineer." One line assigning a title to the model. It is called persona prompting.
And yet the recent academic benchmarks keep saying the same thing. The technique does not work as well as we want it to. In some cases it actively makes the answers worse.
What the benchmarks reveal
A December 2025 paper (arXiv 2512.05858, Playing Pretend: Expert Personas Don't Improve Factual Accuracy) measured the effect of expert personas on two standard factual benchmarks: GPQA Diamond and MMLU-Pro. The result was simple. The accuracy difference between persona and no-persona was either not statistically meaningful, or, where it was meaningful, it was negative. On MMLU, every expert-persona variant dropped the score — an average of 68.0% versus 71.6% with no persona.
Earlier work pointed in the same direction. arXiv 2311.10054 (When "A Helpful Assistant" Is Not Really Helpful) put 2,410 factual questions to four LLM families. Personas produced no meaningful gain. Another study (arXiv 2603.18507) went further: expert personas help with alignment — getting the model to use the desired tone and stance — but hurt factual accuracy. It is a tradeoff.
Why does this matter? When you say "you are a doctor," what you actually want is not a doctor's tone but a doctor's level of accuracy. The tone improves; the answer stays the same or gets worse. Putting the costume on does not change the actor inside.
Why this happens
A theatrical analogy makes it intuitive. The instant you tell the model "play the part of a doctor," it pulls in two kinds of material at once: actual medical knowledge, and the novels, dramas, and screenplays in which doctors appear. arXiv 2406.17260 (Mitigating Hallucination in Fictional Character Role-Play) describes this exact mechanism — character hallucination, where the model produces responses that step outside the character's identity and knowledge boundaries. A doctor persona summons material that imitates a doctor along with the real thing.
More importantly, the persona itself does not increase the model's information. If the model has 1 unit of doctor knowledge, with persona it is 1 and without persona it is 1. The title does not amplify what the model already knows. It just retrieves that knowledge in "a doctor's voice."
This is where users routinely misread the situation. The assumption goes: "power users all use personas; if I imitate them, I'll reach their level." In reality, what OpenAI and Anthropic learn from power users is not how they assign titles. They are tracking a different layer of patterns entirely.
What the labs are actually learning
Look into the RLHF (reinforcement learning from human feedback) pipelines of the major model labs and one thing is clear. The advanced-user data they collect is not about "what persona was used to start." It is about "which facts were handled in which order, in which structure." How the question got narrowed. Which sources got checked, and how. How the user backed out when an answer was wrong.
Research on expert elicitation arrives at the same conclusion. Recent papers like arXiv 2602.02752 follow a recognition over recall principle — you don't ask experts to write prompts from scratch. You show them existing prompts and answers and ask them to critique what's wrong and how to fix it. As the critiques accumulate, the work pattern emerges.
The same field has confirmed something else: chain-of-thought (showing reasoning steps), few-shot (providing examples), and retrieval augmentation reliably outperformed expert role mimicry on accuracy. Title imitation is the weakest tool in the kit.
So what are the real tools?
Expert personas pulled MMLU accuracy from 71.6% down to 68.0% (arXiv 2512.05858). In the same period, ontology-grounded RAG (OG-RAG) recorded +55% fact recall, +40% answer accuracy, and +30% faster source attribution (arXiv 2412.15235).
Laying facts beside the model — ontology and RAG
If personas are about pretending to be someone, the other end of the spectrum is laying down what is true beside the model directly. Two technologies do this.
RAG (Retrieval-Augmented Generation) — just before answering, the model pulls relevant documents and looks at them alongside the question. The answer is grounded not in what the model memorized but in what was just retrieved.
Ontology — an explicit map of the concepts, relations, and rules in a domain. Patient — symptom — diagnosis — drug — side effect. A structure that tells the model how facts in a field connect to each other.
A December 2024 paper on the combination — OG-RAG, Ontology-Grounded RAG (arXiv 2412.15235) — measured the following across four LLMs: fact recall up 55%, answer accuracy up 40%, source attribution 30% faster. This is the kind of numerical movement persona prompting never produced.
Industry has already adopted it. Microsoft Research's GraphRAG extracts entities and relations from private documents, builds them into a graph, and references that graph at answer time. Neo4j is pushing a GraphRAG product down the same path as a standard component of enterprise RAG. The fact-heavy fields — law, medicine, finance — lean hardest on this direction (arXiv 2505.00039 — Ontology-Driven Graph RAG for legal norms).
"Isn't that for the big companies?"
This is where the average user typically stops. Building an ontology and standing up a RAG pipeline feels like an engineer's job. For the formal systems, it is. But the same principle has been ported, in miniature, to individual users.
Claude's Projects, NotebookLM, ChatGPT's Custom GPTs are essentially personal RAG. Drop your domain documents into one of them and the model references those documents first when answering. Not impersonating a title — actually keeping the facts on hand.
The ontology side starts even more simply. Writing down the structure of the facts in your own work — that is, in itself, a small ontology. A doctor: "the patient population I see — main complaints — diagnostic criteria — drugs I commonly prescribe — side effects to watch." One page. A lawyer: "case types I handle — applicable statutes — line of precedents — common client misconceptions." A designer: "design principles I use — reference folders — terms clients commonly misuse and what they actually mean."
Hand that one page to the model alongside every request, and you get a better answer than a persona could produce in a hundred attempts. The reason is plain. The user has filled in, by hand, the regions the model would otherwise have to guess.
Using AI well, in the end, means writing out the fact structure of your domain by hand. A persona line takes five seconds to write, and its effect lasts about that long.
What it actually means to be good at AI
Something becomes clearer at this point. Being genuinely good at AI does not mean having found a magic one-line incantation. It means being able to articulate the fact structure of your own work clearly. That articulation becomes the document inside your RAG, the small ontology, the specification of which information should be handled in which order.
This is not a light task. To write your domain from scratch is to look at your own work from scratch. To distinguish what is genuinely the core from what is conventionally attached. It takes time. It demands persistence. And once it exists, the asset stays — even if the model underneath changes.
A persona prompt takes five seconds to write, and its effect lasts about that long. Articulating the structure of facts in your domain takes days, sometimes months. The effect lasts days and months. To work with AI is, in the end, to do your work more deeply. AI does not do the work for you. AI works better because the human works more precisely.
Two things to try today
One. Write "the one page" for your domain. Five to ten core concepts. Three to five relations between them. Two or three common misconceptions. On your next AI task, drop this page into the system prompt or project context, and compare the answer with what you got from "You are a [profession]" alone. The difference will show.
Two. If you usually start with "You are a specialist in X," remove that line. In its place, write "Facts that must be handled in this task: [list]" and "Things absolutely not to be guessed in this task: [list]". The model works on boundaries and rules, not on titles.
Don't summon the title. Lay down the facts. That is the real pattern of advanced users — the pattern the labs are quietly learning from.
References:
- arXiv 2512.05858 — Playing Pretend: Expert Personas Don't Improve Factual Accuracy
- arXiv 2311.10054 — When "A Helpful Assistant" Is Not Really Helpful
- arXiv 2603.18507 — Expert Personas Improve LLM Alignment but Damage Accuracy
- arXiv 2406.17260 — Mitigating Hallucination in Fictional Character Role-Play
- arXiv 2412.15235 — OG-RAG: Ontology-Grounded Retrieval-Augmented Generation
- arXiv 2505.00039 — An Ontology-Driven Graph RAG for Legal Norms
- Microsoft Research — GraphRAG: Unlocking LLM Discovery on Narrative Private Data
- Neo4j — Unifying LLMs & Knowledge Graphs for GenAI
- Anthropic — Effective Context Engineering for AI Agents
Images in this article were generated with an AI image model (Gemini 3 Pro Image) and edited under the supervision of the YYAIA Tech Desk.
This article was produced with the deep-thinking engine
Simthink Engine (深 Think) of Myeongseong Simjae.