The recommendation in the mean time is to link to the sitemap from a robots.txt file placed in the root of the
server containing the sitemap and metadata. In the robots file, the user agent value indicates the harvest
protocol implemented. For the recommendations above, these are the user agent strings:
Embedded in HTML: User-agent: CDIF-embed-in-HTML
Individual metadata file URLs: User-agent: CDIF-url-get-metadata
Metadata list file: User-agent: CDIF-url-get-metadata-collection
Doug Fils suggests "some of these "hints" might be better done in the sitemap which is more open to extension than robots.txt"
is defining a sitemap XML extension a better solution for this problem. Should we do both? is there a different solution?
ResourceSync (ANSI/NISO Z39.99-2017) has created these extensions already. It allows inclusion of links in sitemaps, see the standard's Linking to Related Resources section.
draft recommendation is
Doug Fils suggests "some of these "hints" might be better done in the sitemap which is more open to extension than robots.txt"
is defining a sitemap XML extension a better solution for this problem. Should we do both? is there a different solution?