workeffortwaste / horseman

The detailed update and issue repository for the Horseman crawler.
https://gethorseman.app/
16 stars 0 forks source link

A number of snippets after first URL return null #102

Closed dwsmart closed 1 year ago

dwsmart commented 1 year ago

A number of the snippets return null after the first URL is tested, for example h1, h1 count, Carbon Footprint etc. I've attached a zip of the JSON export for a crawl of my site.

https_tamethebots.com-2022-10-14T12_11_32.480Z.zip

I quicklt recreated the H1 test (or at least the same results)

/* Retrieve some information from the page */
const result = document.getElementsByTagName('h1').length

/* Return the result to Horseman */
return result

and the count one

/* Retrieve some information from the page */
const result = document.getElementsByTagName('h1').length

/* Return the result to Horseman */
return result

And these do seem to populate throughout the whole crawl.