search

pgh-public-meetings / city-scrapers-pitt

Pittsburgh City Scrapers: sourcing public meetings in Pittsburgh
https://pgh-public-meetings.github.io/events/
MIT License
19 stars 66 forks source link

[Sentry] [pa_development] website changed #173

Closed bonfirefan closed 3 years ago

bonfirefan commented 3 years ago

Error: [pa_development] JSONDecodeError('Expecting value: line 3 column 1 (char 4)') It looks like the URL used is no longer valid: https://dced.pa.gov/wp-json/tribe/events/v1/events Worth digging for the new JSON source

ben-nathanson commented 3 years ago

Weird! This appears to conform to the Wordpress Events Calendar Rest API, and "https://dced.pa.gov/wp-json/tribe/events/v1/events" does return a 200, but the content of the entire page is literally '\r\n\r\n'. Whereas it used to be a giant JSON blob that we could parse into events.

Another wrinkle in things is that I can't find the new meetings page publicly. Not clear what the path forward here is until we can find if/where new meetings are being posted.

ben-nathanson commented 3 years ago

Events still show up here. Perhaps the raw events JSON is only available internally now? Assuming the server is now populating the calendar before sending it to the client.

ben-nathanson commented 3 years ago

It seems like there's a JSON blob for a single month being embedded within the page, which may confirm my idea that the server is grabbing a subset of the data and passing it on. Basically this should be a small (if annoying) fix on our end. Still hopeful I won't have to scrape one month at a time.

bonfirefan commented 3 years ago

@mishugana pointed out that the old link appears to be working. It's possible that it breaks occasionally, but seems to be up so I will close this issue.