biglocalnews / court-scraper

Scrapers for U.S. county court sites.
ISC License
57 stars 18 forks source link

implement fake_useragent headers to solve #154 #155

Open ryanelittle opened 2 years ago

ryanelittle commented 2 years ago

This merge will fix the bug causing OSCN errors (#154). It uses the package fake_useragent (https://pypi.org/project/fake-useragent/) to randomize the user agent for each request. The random function uses real world browser usage statistics to mimic a different browser with each request.