Markdown Negotiation Tester

Check whether a public URL can return a text/markdown representation when an AI agent asks for Markdown through the HTTP Accept header.

The tool sends two public HTTP requests to the same URL. It does not sign in, submit forms or execute browser JavaScript.
What a pass means: the same URL can serve normal HTML to a browser and Markdown to a client that explicitly requests text/markdown. This is a technical accessibility signal, not an AI ranking factor.
Two requests, one URL

What this tester compares

Browser-style request
Accept: text/html, application/xhtml+xml...
Agent-style Markdown request
Accept: text/markdown

A negotiated implementation should be able to serve different representations of the same resource without requiring a separate URL for the agent.

What is Markdown?

Markdown is a plain-text format for structured documents. Instead of wrapping every heading, paragraph, link and list in browser-oriented HTML elements, Markdown uses lightweight syntax such as # for headings, - for lists and simple text notation for links and code. The CommonMark specification describes Markdown as a plain-text format for writing structured documents and provides a consistent parsing model.

A Markdown representation does not have to be a physical .md file stored next to every webpage. With HTTP content negotiation, the same URL can return HTML to a normal browser and Markdown to a client that asks for it.

What is Markdown content negotiation?

RFC 9110 defines the HTTP Accept request header so a user agent can express which response media types it prefers. A server can therefore receive Accept: text/markdown and, if supported, return the requested page as Content-Type: text/markdown.

If the representation changes because of the Accept header, the response should also make that variation visible to caches. RFC 9110 defines Vary for this purpose. A response containing Vary: Accept tells an intermediary that the selected representation depended on the client's Accept header.

Why Markdown can matter for AI agents

AI agents frequently need the information in a page rather than its browser presentation layer. HTML can include navigation, scripts, style hooks, repeated interface elements and other markup that is useful to a browser but may be unnecessary for a machine retrieving the main document.

Cloudflare's Markdown for Agents documentation describes an implementation where an AI client requests text/markdown and Cloudflare converts the HTML representation into Markdown at the edge. Cloudflare states that the explicit structure and lower markup overhead can reduce wasted tokens for AI processing. Its current output also strips non-content elements such as navigation, scripts and styles, can preserve useful metadata as YAML frontmatter, and can preserve JSON-LD structured data.

That does not mean Markdown is inherently required for every agent or that HTML has become obsolete. It means a site can offer a second, machine-oriented representation of the same resource without forcing every automated client to perform its own HTML cleanup.

Why this is part of an AI-readiness strategy

Agent readiness is broader than publishing content that an LLM can read. An agent may need to discover a resource, retrieve a useful representation, understand bot policy, locate an API, discover authentication metadata, or perform an action. Markdown negotiation sits specifically in the content accessibility layer.

Use the Site Agent Readiness Checker when you want to audit the broader stack across discoverability, content accessibility, bot access control, protocol discovery and agentic commerce. Use this Markdown Negotiation Tester when you want to diagnose the content-negotiation layer in more detail.

How to interpret the output

The most important result is not the score by itself. A successful Markdown request confirms that the server can return the requested media type. Content-Type: text/markdown identifies the representation. Vary: Accept indicates that caches should treat the response as dependent on the client's media preference. A different body from the HTML request provides additional evidence that real representation selection occurred.

The response-size comparison is useful for understanding markup overhead, but it should not be confused with a universal token count. Different models tokenize text differently. When a server returns Cloudflare's x-markdown-tokens and x-original-tokens headers, the tester shows them separately rather than estimating them itself.

What a failure does and does not mean

If the URL does not return Markdown, that does not mean the site is inaccessible to AI systems. Many crawlers and agents can parse HTML successfully. It means this optional representation has not been detected for the tested URL.

Likewise, passing this test does not make content more authoritative, improve Google rankings, or guarantee that an answer engine will cite the page. Markdown negotiation solves a transport and representation problem. Content quality, relevance, authority, retrieval, indexing and platform-specific behavior remain separate concerns.

Implementation principle: keep the HTML and Markdown representations semantically aligned. Markdown should make the same public information easier to consume, not create a hidden second version of the page for machines.
This tool also supports Markdown negotiation. Request this same URL with Accept: text/markdown, or open the browser-testable Markdown representation. The query-string version is sent with a noindex header so the canonical HTML page remains the search target.

Primary technical references

Frequently asked questions

The tester requests the same public URL twice: once preferring normal HTML and once with Accept: text/markdown. It checks the HTTP status, Content-Type, Vary header, whether the representations differ, whether the Markdown response still looks like HTML, and the relative response size. It also reports Cloudflare token headers when they are present.

Markdown is a plain-text format for writing structured documents. Headings, lists, links, code and emphasis can be represented with lightweight text syntax instead of large amounts of HTML markup. CommonMark provides a widely used specification for Markdown syntax.

No. HTTP content negotiation can serve different representations from the same URL. An agent can request text/markdown in the Accept header while a browser continues receiving text/html. Some sites also publish separate Markdown URLs, but that is a different implementation choice.

RFC 9110 defines the Accept request header as a way for a user agent to express preferred response media types. A server that supports Markdown can use that preference to return text/markdown instead of text/html for the same resource.

When a response changes based on the Accept request header, Vary: Accept tells caches that the representation depends on that header. This helps prevent a cache from serving the Markdown representation to an HTML browser, or the HTML representation to a client that requested Markdown.

Markdown keeps document structure explicit while removing much of the browser-oriented markup found in HTML. Cloudflare describes this as reducing unnecessary token overhead for AI systems. A cleaner representation can make retrieval and parsing simpler, but it does not guarantee better answers or citations.

No. Agents can and do consume HTML. Markdown negotiation is an optional content-accessibility capability. It becomes more useful when a site wants to expose a predictable, lower-overhead representation to automated clients. It should be considered alongside discoverability, bot policy, APIs, authentication and other agent-readiness layers.

No. There is no published Google, OpenAI, Anthropic or other major platform ranking factor that awards visibility for serving a Markdown representation. The tool measures technical content accessibility, not search or answer-engine ranking probability.

A high score means the URL successfully returns a Markdown media type when requested, signals Accept-based variation to caches, and provides a representation that is distinct from the normal HTML response. It is a Hota Digital technical heuristic, not a standards certification.

The tool compares the bytes returned for the HTML and Markdown responses. A smaller Markdown response suggests less markup overhead for that request. It is not a model token count because tokenization varies by model. If Cloudflare x-markdown-tokens and x-original-tokens headers are present, the tool reports those separately.

A 406 response means the server could not provide a representation acceptable under the request preferences. For this test, it usually means text/markdown is not supported for that URL or the server is configured to reject that media type.

Yes. The underlying mechanism is standard HTTP content negotiation. Cloudflare provides one implementation that converts HTML to Markdown at the edge, but an origin server, framework, proxy or other CDN can also return text/markdown when the client requests it.

This tool performs a deeper test of one content-accessibility capability: Markdown negotiation. The Site Agent Readiness Checker audits a broader set of signals across discoverability, content accessibility, bot access control, protocol discovery and agentic commerce.

The tester makes server-side HTTP requests only. It does not sign in, execute browser JavaScript, verify semantic equivalence line by line, or judge whether the Markdown is factually complete. A dynamically generated client-side Markdown workflow may therefore not be visible to this test.

Why this output matters

Markdown negotiation shows whether the same public URL can expose a cleaner machine-oriented representation when an automated client requests text/markdown. For AEO and AI-readiness teams, the output helps validate content accessibility, cache-safe negotiation and representation overhead without treating Markdown as an AI ranking factor or a replacement for HTML.

Related AEO tools

View AEO tools
Emerging Protocols Checker
Check OAuth metadata, API Catalog, A2A, MCP, Agent Skills, ARD, auth.md, WebMCP and emerging agentic commerce protocol signals on a public domain.
Site Agent Readiness Checker
Audit discovery, content accessibility, bot controls, agent protocols and commerce signals across established standards and emerging agent-web conventions.
Entity Coverage Analyzer
Analyze named entities, expected-entity coverage, contextual clarity, relationships and schema identity signals.
LLM Chunk Checker
Check page sections for retrieval readiness, standalone context, answer clarity and estimated token size.