search

workeffortwaste / horseman

The detailed update and issue repository for the Horseman crawler.
https://gethorseman.app/
18 stars 0 forks source link

Some snippets return only for first URL #105

Closed dwsmart closed 2 years ago

dwsmart commented 2 years ago

There's some snippets that still seem to only return for the first result,

Carbon Footprint, Image Size Decompressed, Image Size Transferred` are an example carbon-fp

Crawl JSON export attached: https_tamethebots.comblog-n-bits-2022-10-19T12_00_31.891Z.json.zip

workeffortwaste commented 2 years ago

This was a weird service worker issue. I've not blocked workers in Horseman in line with how other crawlers work.