search

simonw / til

Today I Learned
https://til.simonwillison.net
Apache License 2.0
1.08k stars 85 forks source link

Add a sitemap.xml file #51

Closed simonw closed 2 years ago

simonw commented 2 years ago

Need a new plugin for this, based on https://simonwillison.net/2020/Jan/6/sitemap-xml/

simonw commented 2 years ago

https://til.simonwillison.net/sitemap.xml works now.

simonw commented 2 years ago
image
simonw commented 2 years ago

Oh interesting - https://til.simonwillison.net/robots.txt now returns:

User-agent: *
Disallow: /tils

That's because datasette-block-robots is also installed.

Interesting that the first time round it was 500 erroring from the datasette-sitemap plugin, but now it's doing the other thing. Reminder that plugin execution order is not guaranteed!

https://github.com/simonw/til/blob/66dd3946c7a14e620b455de27654e2f60ea1d033/metadata.yaml#L10-L13

simonw commented 2 years ago

https://til.simonwillison.net/robots.txt now does:

Sitemap: https://til.simonwillison.net/sitemap.xml
User-agent: *
Disallow: /tils