search

benpickles / parklife

Render a Rack app (Rails/Sinatra/etc) to a static build so it can be served by Netlify, Now, GitHub Pages, S3, or any other web server.
https://parklife.dev
MIT License
193 stars 7 forks source link

Idea: using sitemap for crawling #95

Open Spone opened 1 year ago

Spone commented 1 year ago

Hi! I'm super excited about this project, since it's pretty similar to an idea I was keeping in the back of my mind. So I'm delighted to find an existing gem that implements it!

One idea I had was using the sitemap as a starting point for crawling the site, since sitemaps are usually a pretty comprehensive list of existing URLs.

What are your thoughts on this?

Related:

benpickles commented 1 year ago

That's a good idea - and I can imagine how it might work within Parklife - but at the moment there's no way to hook into the inner workings so it's not possible.

In the meantime, if you have content that you want to be publicly accessible but isn't reachable via crawling I suggest adding a /sitemap.html which was quite common in the old days.