search

biglocalnews / warn-scraper

Command-line interface for downloading WARN Act notices of qualified plant closings and mass layoffs from state government websites
https://warn-scraper.readthedocs.io
Apache License 2.0
28 stars 10 forks source link

GA scraper needs explicit timeouts #586

Open stucka opened 9 months ago

stucka commented 9 months ago

Probably most of the scrapers do, but: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='www.tcsg.edu', port=443): Max retries exceeded with url: /warn-public-view/ (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fa557ff2f70>: Failed to establish a new connection: [Errno 110] Connection timed out'))

Shut down after six hours, looks like