Open anjackson opened 7 years ago
For the sitemap processor, we get this report:
Processor: uk.bl.wap.modules.extractor.RobotsTxtSitemapExtractor 0 links from 4297 CrawlURIs
which is probably because the way we have to enqueue the links means they don't get auto-counted.
Similarly, we should report some worked/failed stats in the Wrenderer.
For the sitemap processor, we get this report:
which is probably because the way we have to enqueue the links means they don't get auto-counted.
Similarly, we should report some worked/failed stats in the Wrenderer.