search

biglocalnews / warn-scraper

Command-line interface for downloading WARN Act notices of qualified plant closings and mass layoffs from state government websites
https://warn-scraper.readthedocs.io
Apache License 2.0
29 stars 10 forks source link

FL scraper down #553

Closed stucka closed 1 year ago

stucka commented 1 year ago

Florida's scraper hasn't picked up any 2023 records yet.

Problem is likely because of the code searching for a base URL with an http prefix vs. the https used in the latest year. The whole approach is weird, though.

Safer way is to mimic the approach used in the HI scraper -- find the right section, then grab all the links in there. Also reasonable bet every http link gets shifted to https anyway so maybe it's worth forcing that to eliminate unnecessary redirects.

Code block of interest: div.class="sfContentBlock"

stucka commented 1 year ago

Patched with https://github.com/biglocalnews/warn-scraper/commit/75e725d8c05b7afffb24b07765c7af1c515cbfbd

Not releasing because I'm still waiting on guidance for Hawaii.