mileswwatkins / Mouse

A chatbot to provide information to backcountry hikers via satellite SMS
4 stars 1 forks source link

Get more-up-to-date PCTA closures data #6

Open mileswwatkins opened 2 years ago

mileswwatkins commented 2 years ago

It appears that the PCTA website, the only source for closure and hazard notices (not apparently republished on Twitter, etc), now has Cloudflare/hCAPTCHA anti-bot code.

I'm communicating with the PCTA tech lead to see if there's a better/another way to access this data, but for now I'm scraping the Google cache (usually ~24–48 hours old) for this information. A longer term solution could be a daily/hourly AWS Lambda using Puppeteer headless Chrome to try avoiding Cloudflare bot detection, or a paid service to get past the hCAPTCHA. Neither are particularly desirable paths; let's hope that the PCTA has an official solution.