search

ClevelandClinicQHS / sociome

Helping researchers to operationalize social determinants of health data.
Other
30 stars 4 forks source link

Block group failures, Census API Issue #2

Closed atperzynski closed 5 years ago

atperzynski commented 5 years ago

The package will regularly fail to extract the idea at the block group level. This appears to be due to a problem with Census API. The error message that the request fails with is, "Error: Your API call has errors. The API message returned is There was an error while running your query. We've logged the error and we'll correct it ASAP. Sorry for the inconvenience."

atperzynski commented 5 years ago

Note that this failure occasionally also appears to occur with census tract level data.

UNFPAmaldives commented 5 years ago

I'm having the same issue.

NikKrieger commented 5 years ago

The main reason this was occurring was that the tidycensus functions upon which get_adi() was based (viz., get_acs() and get_decennial()) usually broke tract- and block group-level calls by county. They would recursively call themselves one county at a time, and if even one county suffered a poor connection error, the entire call would quit.

Starting with the newest version 1.0.1, get_adi() applies the function purrr::insistently(rate = purrr::rate_delay()) to all calls to the Census API. This causes the tidycensus function to try each county again and again until success.

If there are any other problems with tract- or block group-level calls, please describe them, but this fixes the main problem at hand.