workeffortwaste / horseman

The detailed update and issue repository for the Horseman crawler.
https://gethorseman.app/
16 stars 0 forks source link

User-Agent string passed within JSON on startup seemingly not used when crawling #97

Closed chrishaensel closed 1 year ago

chrishaensel commented 1 year ago

When using a headers.json on startup to pass my custom headers for crawling, Horseman apparently does not take the user-agent string I am passing.

my headers.json file: { "User-Agent": "MY-CUSTOM-UA" }

Starting Horseman with horseman.exe --headers="C:\Users\holyshit\Desktop\headers.json"

workeffortwaste commented 1 year ago

Using a custom headers file will now disable device emulation.