adobe / helix-html-pipeline

A library for rendering the html response in Helix3.
https://www.hlx.live/
Apache License 2.0
13 stars 16 forks source link

[helix5] add newline at end of default robots.txt and sitemap.xml #600

Open royfielding opened 4 months ago

royfielding commented 4 months ago

Description The generated default robots.txt and generated sitemap.xml files do not have a newline at the EOF, which can cause problems with poorly written text-based parsers.

To Reproduce

curl https://www.gjfielding.com/robots.txt
curl https://www.gjfielding.com/sitemap.xml

Expected behavior Generate a \n at end of last line.

Additional context The code in src/robots-pipe.js:

  const txt = [
    'User-Agent: *',
    'Allow: /',
    '',
    `Sitemap: https://${prodHost}/sitemap.xml`,
  ].join('\n');

should either have '' at end of list or be a literal like

  const txt = `User-Agent: *
Allow: /

Sitemap: https://${prodHost}/sitemap.xml
`;

maybe (I don't have much javascript language experience).

The same join technique is used in sitemap-pipe.js and can be fixed in same way.

tripodsan commented 4 months ago

thanks @royfielding . I think adding the '' empty line at the end of the array is the easiest to adjust. pull request welcome :-)