janus/sitemaps

Janus generates XML sitemaps from the entity stream: always current, no cron jobs, no object storage — rendered on request from Rama pstates.

The implementation is the janus/sitemaps component (components/sitemaps), exposing janus.sitemaps.api and janus.sitemaps.specs. This page is the wiring guide; the vocabulary and internals live next to the code in components/sitemaps/doc/.

The mental model: sections define what sitemap content exists (module side, Rama), composition defines how it is published (facade side, configuration). SVE is the running example throughout — it publishes two indexes per tenant under the legacy sitemap-service URLs.

1. Declare the sections in your module

One sitemap-sections call, fed from an entity-update depot. Pick a key per section — every module-global name (topology, queries, pstates) derives from it.

(sitemap-sections setup
                  topologies
                  :input-depot  *updated-entities
                  :origins-attr :janus.meta.origin/available
                  :sections     (:sitemaps config))

with the section configuration in the module’s config map:

:sitemaps
{:articles {:type                   :article-months
            :bucket-size            32
            :exclude-route-prefixes #{"/themen/"}}
 :news     {:type                   :google-news
            :publication-names      {:nordkurier.de   "Nordkurier"
                                     :schwaebische.de "Schwäbische Zeitung"}
            :tags                   #{:ld.article/regular}
            :exclude-route-prefixes #{"/themen/" "/ratgeber/"}}
 :authors  {:type          :authors
            :author-tag    :ld.data-record/author
            :author->entry `sve-authors/author-sitemap-entry}}

What to know here:

  • :origins-attr names the entity attribute holding the origin(s) — a set (SVE) or a single value (CHM-style); entities available on several origins are listed in each origin’s sitemaps.

  • Each section type has its own required keys (spec-checked at module assembly — a bad config fails the deploy, not a request): :article-months needs :bucket-size; :google-news needs :publication-names; :authors needs :author-tag and :author→entry.

  • :exclude-route-prefixes is complete per section — the news set does not inherit the articles set, so SVE passes the union there.

  • :author→entry is a qualified symbol of a fn (author record → url-entry map, nil = unlisted); it is resolved on the Rama workers.

  • Entities must carry :janus/tag (isa? :ld.document/article for the article-driven sections), :janus.meta/updated, :janus.meta/routes and the origins attribute; :janus.meta/no-index excludes.

This declares, per section <key>: a sitemap-<key> microbatch topology, its pstates ($$sitemap-<key> …) and two query topologies — sitemap-<key>-index and sitemap-<key>-entries.

2. Register the section queries in the facade config

The facade reaches the sections through foreign queries, under exactly those derived names:

;; in the [:rama/foreign …] :queries map
:rama.module.sve/sitemap-articles-index   "sitemap-articles-index"
:rama.module.sve/sitemap-articles-entries "sitemap-articles-entries"
;; … same pair for every section

3. Compose the published sitemaps

The composition block is the whole public shape: which named indexes exist, which sections they contain (in order), and what URLs everything gets. Templates are two-way — they render the leaf URLs listed in an index and route incoming requests back to (section, params), so the URL scheme lives in exactly one place.

:io.forward-publishing.janus.facade.routes.sitemaps/routes
{:delivery-config #ig/ref :io.…facade.service/delivery-config
 :origin->tenant  {:schwaebische.de "sch" :nordkurier.de "nord"}
 :origin->image-host #profile {…}   ; author-image CDN per env, nil = as-is

 :sections
 {:articles {:index-query   #ig/ref :rama.module.sve/sitemap-articles-index
             :entries-query #ig/ref :rama.module.sve/sitemap-articles-entries}
  :news     {…}
  :authors  {…}}

 :sitemaps
 [{:name       "google-news"
   :index-path "/sitemaps/{tenant}/google-news-sitemap-index.xml"
   :sections
   [{:section     :news
     :entry-paths ["/sitemaps/{tenant}/google-news-sitemap-{bucket}.xml"]}]}
  {:name       "organic"
   :index-path "/sitemaps/{tenant}/organic-sitemap-index.xml"
   :sections
   [{:section     :articles
     :entry-paths ["/sitemaps/{tenant}/organic-sitemap-{yyyy}-{M}.{bucket}.xml"]}
    {:section     :authors
     :entry-paths ["/sitemaps/{tenant}/organic-sitemap-authors.{bucket}.xml"]}]}]}

What to know here:

  • Template placeholders: {yyyy} {MM} (zero-padded) {M} (unpadded) {bucket}, plus one routing placeholder per template — {tenant} (resolved through :origin→tenant, which must be bijective) or {origin} (the origin host itself).

  • Section order inside a sitemap is the index order — SVE’s authors sitemap appears inside the organic index because it is listed second.

  • Alternative :entry-paths per section are tried in order when rendering (a template only renders when all its placeholders have params — that is how CHM’s bucketless single-bucket months would work).

  • The configuration is validated at facade start: composition spec, template compilation, and a round-trip check that no template shadows another. Requests that match no template, an unknown tenant, or a non-canonical URL variant (e.g. a zero-padded month in an {M} template) get a 404.

4. Point robots.txt at the index paths

One Sitemap: line per composed sitemap, e.g. https://www.nordkurier.de/sitemaps/nord/google-news-sitemap-index.xml. robots.txt itself is owned by the delivery app, not by janus.

That’s all

No route code, no XML code, no per-client handler: the facade ns registers one catch-all route per URL prefix and serves every index and leaf generically from the section queries. Adding a client, another index, or moving a section between indexes is configuration; a new kind of content (say, a video sitemap) is one new section type, immediately composable by every client.

CHM

CHM is still served by the legacy implementation (io.forward-publishing.janus.topologies.sitemap + the routes.sitemap facade ns) — one root index with pages, news and monthly article sitemaps. Its migration to the component (in-place Rama pstate migration, or reindex-driven repopulation — the update depots are trimmed, so there is no full-history replay) is a future step; see the component’s ADR 0001.