A sitemap of all the threads would help crawlers and improve search engine discovery. You can point crawlers to the sitemap using the Sitemap: <full-url> directive in a robots.txt file, or by sending them a ping. A sitemap can have a maximum of 50,000 URLs. Beyond that, you need to generate a sitemap index file that points to the different sitemaps.
Not sure if this is a rib issue or a zulip-archive issue, but you could leverage this package I published recently to build the sitemaps and sitemap index: https://hackage.haskell.org/package/sitemap-gen
A sitemap of all the threads would help crawlers and improve search engine discovery. You can point crawlers to the sitemap using the
Sitemap: <full-url>directive in arobots.txtfile, or by sending them a ping. A sitemap can have a maximum of 50,000 URLs. Beyond that, you need to generate a sitemap index file that points to the different sitemaps.Not sure if this is a
ribissue or azulip-archiveissue, but you could leverage this package I published recently to build the sitemaps and sitemap index: https://hackage.haskell.org/package/sitemap-gen