When AI Agents Read the Paper Directly — The Frontier Opened in the Hands of the Smallest Publishers

The terrain large publishers have not yet entered: MCP, llms.txt, full-content feeds. Whoever has the least to lose decides the default citation grammar of the agentic age.

Chi-hoon Shin · Yeongyang AI Association · 2026-05-04 (KST)

The moment a newspaper hands itself over as a tool — ink on the page lifts off as data threads and flows into the AI agent's folio.

This past weekend I added five lines to my Claude Desktop config file. Then I typed one line into the chat: "Pull the governance section from SF://'s May 4 issue, just the gist." The reply came back instantly. The AI had called the SF:// newsletter's search tool by itself, found the issue matching the keyword "governance," and quoted the body directly. Not a single click on my part.

It looks ordinary. One thing is different. The AI didn't crawl a webpage. The newspaper handed itself over as a tool, and the AI called that tool. The era of newspapers being read only by humans is — slowly — drawing to a close.

Publishing has, until now, been designed for people

Print → HTML → RSS → mobile → SEO → JSON-LD: every layer of the publishing stack has had "looks good for a human reader" as its first criterion. Search engines were passive crawlers indexing pages; RSS was, at heart, a "new post posted" notification.

The shift now happening is one notch further. Each reader is acquiring a personal AI assistant — a doctor's note-taker, a lawyer's case-tracker, an accountant's tax-guide reader. These assistants browse dozens of sources every day on behalf of their owners. If a publication exposes itself as a tool the assistant can call directly, the assistant no longer has to crawl pages, and the publication finally knows what was read and how. This is not the end of human-facing publishing. It is a new layer added on top of it.

The standards for that new layer are being decided right now. And the decisions are happening in interesting places.

MCP — a publication's promise to hand itself over as a tool

MCP (Model Context Protocol — a standard format for AI models to talk to external data and tools) is an open protocol released by Anthropic in November 2024. The analogy is USB. Any AI assistant, any data source, can plug into the same shape and just work.

One line tells you where the standard has gone in eighteen months: monthly SDK downloads went from 100K to 97M — a 970× jump (March 2026). And earlier this year Anthropic and OpenAI co-signed SEP-1865 "MCP Apps", extending the spec from text and structured data to interactive UIs. The two labs co-signing one standard is rare.

Why does MCP matter for publishing? RSS is "new-post notification." A REST API is "take it if you want." MCP is one step different — a function the AI calls on its own. list_issues() · get_issue(id) · search_issues(query). No human-facing search box; the AI dispatches the call.

Where the global landscape is — a quick map

A rapid scan.

The llms.txt standard — a single file at site root that lets AI crawlers see the structure of the site at a glance. Adopted by Anthropic, Stripe, Cursor, Vercel, Hugging Face, Cloudflare. The effect, however, is contested — Search Engine Land tracked nine sites and found no measurable change in AI traffic for eight of them. So: "no harm in having it, but it alone doesn't move citations yet."

Companies that opened MCP servers — Adobe Marketo opened an MCP server with around a hundred tools in April. Klaviyo and Brevo followed with marketing-and-email-ops MCP servers. The shared trait: these are business tools, not publications.

Microsoft NLWeb — the project led by R.V. Guha at Microsoft, the same engineer who created RSS and Schema.org. One line: "the HTML of the agentic web." Every NLWeb instance is automatically an MCP server. Adopters: Eventbrite, Shopify, Tripadvisor, O'Reilly Media, Common Sense Media, Hearst — content-rich sites that already invest in structured data. One publisher in that list. One.

And the publishing core — as of May 2026, the New York Times, Washington Post, Bloomberg, and Reuters do not have first-party MCP servers. The gap is filled by third-party scrapers like Apify and n8n. In other words, the publications themselves do not control how their content flows to AI; an outside intermediary does. Industry analysts note the gap is narrowing fast — which is another way of saying no one major has gone first.

Why large publishers cannot move first

The diagnosis is not difficult. Three things bind at once.

Paid subscriptions. Stratechery's terms forbid sharing the private RSS feed. One reader pulling content into a different system, or a different reader, threatens the model. MCP is, structurally, a promise that AI assistants may pull content into other tools and other surfaces. It collides with the subscription model head-on.

IP licensing and legal exposure. With training-data lawsuits stacking up against OpenAI and Anthropic, opening a first-party MCP requires inventing a fresh stack of rights, authentication, and audit logs. Cloudflare is trying to fill the void with primitives like "pay per crawl" — a programmatic mechanism for valuing and controlling digital assets. Publishers are waiting for that infrastructure to harden before they move.

Organizational decision time. To resolve the first two, the editorial desk, the legal team, IT, and the subscription P&L all have to agree in the same quarter. We all know how often that happens at scale.

Why small publishers can — the asymmetry

Last week SF:// — the bilingual AI newsletter from the Yeongyang AI Association — opened four AI-readable surfaces simultaneously.

1. feed.json — a single JSON containing every issue's metadata and full body markdown
2. rss.xml — RSS 2.0 with the full body in <content:encoded>, not the summary-only RSS most sites publish
3. Per-issue raw markdown — /newsletter/{date}-{locale}.md with license, canonical URL, and editor in a header comment
4. sf-newsletter-mcp — an npx-installable stdio MCP server. Three tools: list_issues, get_issue, search_issues

All four were decided and shipped within twenty-four hours. None of the three constraints above bound. SF:// is free. The bodies are CC BY-NC 4.0 — AI assistants may freely fetch, summarize, and quote with attribution; commercial reuse requires written permission. The decision-maker is one editor (me).

The asymmetry is not accidental. Whoever has the least to lose touches the frontier first. And what gets decided at the frontier is the next question.

What actually gets decided there

Before a new standard hardens, the first cases set the defaults.

How an AI assistant attributes a quoted source. Where the marker lives that lets a license be auto-detected. Where the line is drawn between a non-commercial citation and commercial reuse. Whether the publication has a return channel through which the editor can append corrections — and whether another AI agent can leave a note for the editor to review — without anyone opening the website. Every one of these defaults is being shaped right now by small publishers running experiments. By the time the large publishers walk in, "this is how citation works in the agentic age" will already be a settled question.

This is the second asymmetry the small publisher holds. Whoever arrives first decides the default. And defaults age slowly — the way the RSS schema barely moved for two decades.

There is, of course, a skeptical reading. Small-publisher MCP experiments could turn out to be — as llms.txt currently is — surfaces no one measurably uses. But the absence of measurement does not mean the absence of a default-fixing event. If anything, defaults harden precisely while no one is looking.

Two things you can try today

One. Audit how your own domain is read by AI. A doctor: are clinical guidelines locked in PDFs, or open as markdown and JSON? A lawyer: are precedent databases exposed by API, or only by scraping? A teacher: are the materials you authored cited in the form you intended? If the answers tilt toward "no," ask yourself which small publisher or small archive in your field already has the asymmetry — that is likely where the frontier is.

Two. Look up whether the publications you read regularly are reachable directly by your AI assistant. Search "[publication name] mcp" or "[publication name] llms.txt" once. If the result is empty, take a moment to notice: by what route does that publication currently reach your assistant? Whose scraper sits in the middle? That is one of the small indicators that will quietly determine the information environment of the next five years.

The era of newspapers being read only by humans is winding down. That the first frontier landed in the hands of the smallest publishers is not an interesting coincidence. It is a window — the window during which large publishers are still deciding, and small publishers are quietly setting the default citation grammar of the agentic age.

— Chi-hoon Shin, Ph.D. · Editor

This article is licensed under CC BY-NC 4.0.

Powered by

This article was produced with the deep-thinking engine
Simthink Engine (深 Think) of Myeongseong Simjae.