tiefenb
commented
1 year ago - based on category sitemap.xml with some logic to index only relevant pages since there are a lot of seo, brand and landing pages
- added basic category mapping
- will run longer (5-8 minutes) since it crawls some hundred lister pages which respond a little bit slower with sz=1000 products
badlogic