search

wpinvestigative / arcos

https://wpinvestigative.github.io/arcos/
Other
30 stars 18 forks source link

Timeout for certain counties? #2

Closed jeffcsauer closed 3 years ago

jeffcsauer commented 4 years ago

Hi all,

when using county_raw(), some counties appear to time out and report a HTTP error 502. Can any others replicate on their machines? I can provide a list of all the counties where this appears to be an issue.

Example: library(arcos) test = county_raw(county="King", state="WA", key="WaPo")

This occurs for about 200 counties.

Edit: another quick example with LA:

> test = county_raw(county = "los angeles", state = "CA", key = "WaPo") Error in open.connection(con, "rb") : HTTP error 504. > test = county_raw(county = "LosAngeles", state = "CA", key = "WaPo") Error in open.connection(con, "rb") : HTTP error 502. > test = county_raw(county = "Los Angeles", state = "CA", key = "WaPo") Error in open.connection(con, "rb") : HTTP error 504.

andrewbtran commented 4 years ago

Thanks for letting us know. We're looking into bumping up the timeout limit to accommodate huge files like those counties.

jeffcsauer commented 3 years ago

Hi @andrewbtran - any update on this? Still getting an HTTP error 502 when using county_raw() on LA. Do you think it would be better to just handle the raw data for the time being?

andrewbtran commented 3 years ago

Finally fixed. The original API has been updated to return a csv link instead of presenting as json. It required changing the ARCOS R package function to read in as a csv with vroom instead of reading in as a json file. The dev version has been updated. Will push to CRAN next week.

jeffcsauer commented 3 years ago

Thanks so much, @andrewbtran! This is so great!!!

jeffcsauer commented 3 years ago

Can confirm, counties like Los Angeles are now successfully downloading!