Generated files

Every build also writes sitemap.xml, llms.txt, llms-full.txt, per-page Markdown, and a full-text search index.

Every vark build writes these automatically:

`sitemap.xml`

A standard sitemap of every page, using baseUrl from aardvark.config.yaml.

`robots.txt`

Allows all crawlers, points them at the sitemap, and declares Content Signals — a machine-readable statement of how automated systems may use your content, per contentsignals.org and its IETF draft. A short comment block explains the policy, and one Content-Signal: line carries it:

User-agent: *
Content-Signal: search=yes, ai-input=yes, ai-train=yes
Allow: /

The three signals are:

search — building a search index and showing links and short excerpts in results (not AI-generated summaries).
ai-input — using the content in AI models in real time (retrieval-augmented generation, grounding, generative search answers).
ai-train — training or fine-tuning AI models.

Each is yes or no. By default aardvark grants all three. To change that, add a top-level robots: block — distinct from seo.robots, which is the per-page <meta name="robots"> directive:

robots:
  contentSignals:
    search: yes      # listed signals override; unlisted ones stay at the yes default
    ai-input: yes
    ai-train: no     # reserve training rights while still allowing search + AI answers

Listed signals override the default; an unlisted signal stays yes. Set one to no to deny that use, or to null to drop it from the line (declaring no preference — neither granting nor restricting it).

To omit Content Signals entirely, set robots: false (or robots: { contentSignals: false }) — only an explicit false (or off/0) disables. A bare or empty contentSignals: is read as “not configured” and falls back to the all-yes default, so a placeholder key never silently strips the signals. Either way the rest of the robots.txt is still written.

As with every artifact here, a hand-written static/robots.txt (or public/robots.txt) is copied verbatim and always wins — aardvark won’t overwrite it.

`llms.txt` and `llms-full.txt`

Per the llmstxt.org convention, to make your docs easy for LLMs to consume:

llms.txt — an index: your site name, summary, and a linked list of every page with its description. Each entry links to the page’s raw .md (below) so an assistant gets clean Markdown, not rendered HTML — or to the HTML page when per-page Markdown is turned off.
llms-full.txt — the full content of every page concatenated, with island markup stripped to clean text.

site.summary (or site.description) in your config becomes the llms.txt summary; each page’s description is used in the index list.

`search-index.json`

A full-text index — one record per page (URL, title, breadcrumb, description, keywords, headings, inbound link text, and body text) — that powers the built-in search. Always written unless you set search: false.

These need no configuration beyond baseUrl and your page frontmatter.

Web Bot Auth directory

When you enable webBotAuth, the build also publishes a key directory (a JWKS) at /.well-known/http-message-signatures-directory, so your site can identify itself when an agent sends signed requests on its behalf. It’s off until you add a key — see Web Bot Auth.

`.well-known/agent-skills/index.json`

If your project has a skills/ directory, every skills/<name>/SKILL.md is published as an Agent Skills Discovery index at /.well-known/agent-skills/index.json, per the Agent Skills Discovery RFC (v0.2.0). Agents that support the convention can then discover the skills your site offers — the way llms.txt and sitemap.xml advertise your content.

The index lists each skill with its name, type, description, URL, and a SHA-256 content digest:

{
  "$schema": "https://schemas.agentskills.io/discovery/0.2.0/schema.json",
  "skills": [
    {
      "name": "deploy-a-docs-site",
      "type": "skill-md",
      "description": "Build an aardvark documentation site and deploy it to a static host.",
      "url": "/.well-known/agent-skills/deploy-a-docs-site/SKILL.md",
      "digest": "sha256:…"
    }
  ]
}

Each SKILL.md is copied verbatim to /.well-known/agent-skills/<name>/SKILL.md (the <name> is its source directory), and its digest is the SHA-256 of that file’s bytes, so a client can verify what it fetched. The skills come from your skills/ directory — generated by vark ai-enrich or hand-authored; either way vark build just publishes whatever is on disk, with no AI configuration required. The build also synthesizes a “how to use this site” agent-help skill, so the index is published on every build whenever the Markdown menu is on (the default) — even with no skills/ directory. It is omitted only when there are no skills at all (no skills/ directory and markdownMenu off).

As with every artifact here, a hand-written static/.well-known/agent-skills/index.json always wins — aardvark then leaves the whole .well-known/agent-skills/ tree untouched.

Agent discovery — `Link` headers

So an AI agent landing on your site can find the machine-readable files above, every build sets an RFC 8288 Link response header on the home page (/). It’s written into the same _headers file, so it works on hosts that read it (Cloudflare Pages, Netlify) and under vark serve, and points at each resource with a registered relation type — all on one comma-separated header:

</llms.txt> and </llms-full.txt> — rel="service-desc"
</sitemap.xml> — rel="describedby"
</metadata.json> — rel="service-desc", when the AI assistant or MCP is on and markdownMenu is enabled (the build only writes metadata.json when per-page Markdown is published)
</.well-known/agent-skills/index.json> — rel="service-desc", whenever the agent-skills index is published (with the Markdown menu on, that’s every build — above)
</.well-known/api-catalog.json> — rel="api-catalog", when the site has an OpenAPI reference (below)

No configuration. To override it, declare a / block in your own static/_headers. A site-wide seo.noindex (a staging build) suppresses it, matching how noindex keeps a build out of every other discovery surface.

`api-catalog` for OpenAPI sites

If any page embeds an {% openapi %} reference, the build also writes an RFC 9727 catalog at /.well-known/api-catalog.json — an application/linkset+json document listing each API with a machine-readable service-desc link to the spec (republished as JSON under /_aardvark/openapi/) and a service-doc link to its docs page. A request to the canonical extensionless /.well-known/api-catalog redirects there. Nothing to configure — it appears only when you actually have a spec.

Per-page Markdown and the “View in Markdown” button

Every page is also written as raw Markdown beside its HTML, so readers — and LLMs — can grab the source. A page at /guide/intro/ is served at /guide/intro.md; the home page is /index.md. It’s the same processed Markdown that feeds llms-full.txt, with island markup stripped to clean text.

To surface it, each page shows a View in Markdown button at the top-right of the content. Clicking the label opens the page’s .md; the chevron beside it opens the rest:

Copy page — copy the Markdown to the clipboard.
Download PDF — download the whole site as a single PDF. Shown only when pdf: true is set (see below).

The dropdown also carries the agent hand-off items — Copy MCP Server, Install Skill, Install Plugin, and Install Assistant — covered on the Agent readiness page. aardvark also writes a _headers rule so hosts that read it (Cloudflare, Netlify) serve the .md as text/plain — shown inline rather than the default text/markdown, which browsers download — and the generated .txt files (llms.txt, llms-full.txt, robots.txt) as text/plain; charset=utf-8, so characters like em-dashes render correctly instead of as mojibake. Hosts that ignore _headers (Vercel, GitHub Pages, …) still serve the files; add your own header rule there if you want inline display and UTF-8.

Tuning

On by default. A bare markdownMenu: false turns off both the .md files and the button; otherwise toggle individual actions or set the button label:

markdownMenu:
  enabled: true               # gates the .md files AND the button
  viewMarkdown: true          # per-item toggles (all default true)
  copyMarkdown: true
  downloadPdf: true
  label: View in Markdown     # the primary button's label

The agent hand-off items have their own toggles (copyMcp, installSkill, installPlugin, installAssistant) — see Agent readiness.

PDF output

Set a top-level pdf: true to render the whole site to one downloadable PDF — a cover page, a clickable table of contents, then every page in nav reading order with a bookmark per page — named after your site (aardvark → /aardvark.pdf). A Download PDF item is added to the dropdown on every page, linking to that single document. It’s laid out as a clean, printable handbook with syntax-highlighted code (long lines wrap) and tables. Built-in components are re-rendered for print — an {% openapi %} reference becomes operation/parameter tables, a card or callout a titled block, and so on — so interactive widgets aren’t blank on paper; a live map gets a “view online” note, and a component from a React library you import is left to its plain content. Links between your own pages become in-document jumps (so the PDF reads on its own), while links to other sites stay clickable. The cover shows your theme logo, site name, description, base URL and a build timestamp. Password-protected pages are never included (their content ships encrypted). It’s off by default since rendering adds a little build time.

pdf: true

To keep generating the PDF but hide the menu link, set markdownMenu: { downloadPdf: false }. The cover carries a small “Built by aardvark” credit; like the on-page Powered-by footer, it’s removable with poweredBy: false on sites that use aardvark’s AI features.

Reuse the PDF instead of re-rendering every build

The whole-site PDF is the slowest part of a build, so you can let a build reuse the copy already published rather than rendering a fresh one every time. Give pdf as an object with reuseForDays:

pdf:
  reuseForDays: 7

On each build aardvark fetches a tiny stable sidecar at <baseUrl>/_aardvark/pdf-reuse.json to find the last deployed fingerprinted PDF URL (falling back to the legacy <baseUrl>/<slug>.pdf, where the slug is derived from your site.name, e.g. “My Docs” → /my-docs.pdf). It then reads the build date embedded in that PDF itself (its /CreationDate). While that’s younger than reuseForDays, the live PDF is republished as-is and the slow render is skipped; once it ages past the window, the next build renders a fresh one — whose new creation date restarts the clock. Because reuse republishes the exact bytes, the embedded date rides along unchanged, so republishing never resets the window — only a real render does. This needs a baseUrl to fetch from; anything missing or unreadable safely falls back to rendering, so a build never ends up without a PDF. Pass --no-pdf-reuse, or use a bare pdf: true, to always render.

The reuse fetch is locked to the baseUrl host (an SSRF guard — it won’t follow a redirect to a different origin). So if your live PDF is served off-host (e.g. a redirect to an R2/CDN origin), reuse won’t engage and the build just renders fresh; keep the PDF on the baseUrl host to benefit from reuse.

Password-protected pages: a fresh render drops protected pages from the PDF (their content would otherwise ship in the clear), but reuse republishes the live PDF as-is and can’t re-apply that exclusion. So if you newly protect a page that was previously public, deploy once with --no-pdf-reuse (or pdf: true) — otherwise that page’s content lingers in the public PDF until the next fresh render (up to reuseForDays later). When the build reuses a PDF on a site that has protected pages, it notes this on the “Render PDFs” line (it isn’t a warning — on a stable-protection site that would fire every build).

`_redirects` and alias stubs

Keep old URLs working after you move or rename a page. Two mechanisms feed the build — use either or both.

Per-page `aliases:` (front matter)

List a page’s historical paths and each one forwards to the page:

---
title: CLI reference
aliases:
  - /old-cli/          # forwards to /cli/
  - /legacy/cli.html   # an explicit file works too
---

For every alias, aardvark writes a tiny HTML stub at the old path with a rel="canonical" to the real page, robots: noindex, and an instant <meta http-equiv="refresh"> (plus a JS redirect and a visible link). Google treats an instant refresh as a permanent move, and the canonical consolidates the old URL’s ranking onto the new one — so search engines update themselves. Stubs work on every host and in vark dev, and each alias also gets a true 301 line in the _redirects file below.

Site-wide `redirects:` (config)

For rules that aren’t tied to one page — including * wildcards and :slug / :splat placeholders — add a redirects: list to aardvark.config.yaml:

redirects:
  - /blog/* /news/:splat 301        # a raw _redirects line, passed through as-is
  - from: /docs/:slug               # or the mapping form
    to:   /guide/:slug
    status: 301

Each list item is one rule on a single line (a multi-line value is truncated at the first newline). They’re written verbatim to _redirects. A static generator can’t expand a wildcard into files, so — unlike concrete aliases: — they produce no stubs and only take effect on a host that reads _redirects (Cloudflare Pages, Netlify, Vercel). On other hosts they’re inert.

Tuning

aliases:
  htmlStubs: true       # write the per-alias HTML stubs (default true)
  redirectsFile: true   # also emit a _redirects file (default true)
  force: false          # append `!` to alias 301s — Netlify force flag (default false)

A bare aliases: false turns off stub generation entirely. One caveat for true 301s: on Cloudflare Pages and Netlify a static file shadows a _redirects rule for the same path, so an alias’s own stub wins over its 301 there. If you deploy only to those hosts and want header-level 301s, set aliases: { htmlStubs: false } so the _redirects rule is unshadowed.

Generated files

sitemap.xml

robots.txt

llms.txt and llms-full.txt

search-index.json

Web Bot Auth directory

.well-known/agent-skills/index.json

Agent discovery — Link headers

api-catalog for OpenAPI sites

Per-page Markdown and the “View in Markdown” button

Tuning

PDF output

Reuse the PDF instead of re-rendering every build

_redirects and alias stubs

Per-page aliases: (front matter)

Site-wide redirects: (config)

Tuning

`sitemap.xml`

`robots.txt`

`llms.txt` and `llms-full.txt`

`search-index.json`

`.well-known/agent-skills/index.json`

Agent discovery — `Link` headers

`api-catalog` for OpenAPI sites

`_redirects` and alias stubs

Per-page `aliases:` (front matter)

Site-wide `redirects:` (config)