search

ThomasTJdev / nim_websitecreator

Nim fullstack website framework - deploy a website within minutes
https://nimwc.org
MIT License
176 stars 7 forks source link

Include public/files in sitemap.xml #19

Closed ThomasTJdev closed 5 years ago

ThomasTJdev commented 5 years ago

Should the public files be included in the sitemap? This could easily be done by adding them in the genSitemap() with existing proc.

But how should we then handle it, if 1 of the files is deleted - we need to inform the indexer, that the file is gone by responding with a 410. This means, that we need to keep track of files.

The same problem can be found if the user decides to delete a blog post or change the URL.

Should the user then handle this by them self by using .htaccess or a rewrite ^?

Roadmap proposal:

  1. Let user decide, if files should be added to sitemap
  2. When user deletes a page or file, inform about the need to respond with a 410
  3. User can then decide to add the deleted item to table deleted_files_410, which would be loaded into the sitemap
juancarlospaco commented 5 years ago

I think sitemap are for stable permanent pages only, because third parties may want to "Permalink" it.

...And adding literally everything is kinda a too much work and maybe a performance hog too. :thinking:

ThomasTJdev commented 5 years ago

Well you are right. Only static should be included.. it's just because the files I place in public/files are static. But my personal configuration should not influence (too much :wink: ) on NimWC. It could maybe be included as an option?

Regarding the performance: Google's indexer allows up to 50_000 URL's with a size up to 50MB, before you need to split it into multiple sitemaps.

I'm closing the issue, since this is a minor wish from my side, but a larger code task.