Open dmdboi opened 3 months ago
How would you think this would work?
I think at minimum, a robots.txt file is generated if its a web app. Still working out details for sitemaps
I'm trying to figure out how automating these would be good and how, and all I can think of is maybe generating them dynamically based on the route definitions? If they don't have auth middleware on them, we include them in a sitemap.xml
route, and maybe if they do have auth middleware we put them in robots.txt
route as don't visit?
Yeah, I was thinking the same.
Kubit should come with a way to auto-generate sitemaps and robots.txt