walkerke / tidycensus

Load US Census boundary and attribute data as 'tidyverse' and 'sf'-ready data frames in R
https://walker-data.com/tidycensus
Other
639 stars 100 forks source link

Bad request for get_acs() #99

Closed plnnr closed 6 years ago

plnnr commented 6 years ago

Perhaps this is related to the API endpoint changes, but these two code chunks vary only by the year variable. The 2016 and 2015 ACS pull works, but the 2010 (and also 2014) does not.

Works:

hhincome <- get_acs(geography = 'tract', table = 'B19001', 
                    state = "OR", survey = 'acs5', 
                    year = 2016, output = 'wide')

Does not work:

hhincome <- get_acs(geography = 'tract', table = 'B19001', 
                    state = "OR", survey = 'acs5', 
                    year = 2010, output = 'wide')

The exact error reads:

Client error Bad Request Client error: (400) Bad Request
Error in as.character(x) : 
cannot coerce type 'closure' to vector of type 'character'

I install_github-ed the latest version this morning (8/21/2018).

plnnr commented 6 years ago

When I revert to the release version (0.4.6) it resolves the issue.

walkerke commented 6 years ago

Hmm - I do note that your code works for 2010 and 2014 when geography is "county". I'll investigate.

walkerke commented 6 years ago

I'm getting the sense that this is due to the Census Bureau making changes to how data are available. For example:

hhincome <- get_acs(geography = 'tract', table = 'B19001', 
                    state = "OR", county = "Benton", survey = 'acs5', 
                    year = 2010, output = 'wide')

works with the development version, which uses the new endpoints. This suggests that 2006-2010 data for tracts is only available by county for the new endpoint.

walkerke commented 6 years ago

This is confirmed examining the API. For example, https://api.census.gov/data/2015/acs/acs5?get=NAME,B01001_001E&for=tract:*&in=state:01 works, but https://api.census.gov/data/2014/acs/acs5?get=NAME,B01001_001E&for=tract:*&in=state:01 returns error: unknown/unsupported geography heirarchy.

However, https://api.census.gov/data/2014/acs5?get=NAME,B01001_001E&for=tract:*&in=state:01, which uses the old endpoint, works.

I can fix this by stitching together tract data by county for a given state like I do for block groups. However - @loganpowell do you have any insight into why this change was made?

walkerke commented 6 years ago

This is (temporarily) fixed as of https://github.com/walkerke/tidycensus/commit/cff7a92bd8ad8705316eb9c9e134ec88bf551fd0. I iterate through all counties in the state and pull tract data for them, then stitch them together. It is (much) slower but it is a solution that'll work if we no longer can pull all tracts for a state for those years.

loganpowell commented 6 years ago

Sorry for the delayed response. Thank you for bringing the tracts-at-state-level issue to my attention and for emailing cnmp.developers.list@census.gov. The API development team is discussing how to best address this issue and will be returning to let me know. Stay tuned.

On Sun, Aug 26, 2018 at 10:51 PM, Kyle Walker notifications@github.com wrote:

This is (temporarily) fixed as of cff7a92 https://github.com/walkerke/tidycensus/commit/cff7a92bd8ad8705316eb9c9e134ec88bf551fd0. I iterate through all counties in the state and pull tract data for them, then stitch them together. It is (much) slower but it is a solution that'll work if we no longer can pull all tracts for a state for those years.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/walkerke/tidycensus/issues/99#issuecomment-416099823, or mute the thread https://github.com/notifications/unsubscribe-auth/ADQBP64fiUIe9Z0SyqO-3AI-SzKcXporks5uU16sgaJpZM4WGV7U .

walkerke commented 6 years ago

Thanks @loganpowell!

loganpowell commented 6 years ago

So, the team has met and would like you guys to know that we are undergoing technology changes, which - by default - is causing the issues you (and others) are having with the county wildcard, but when the transition to the new stack is complete, we will be patching it to avail the same wildcard functionality to make it as seamless for you as possible, however the patch will take about a month, but should be done and released by the end of September. We hope this isn't too great an inconvenience in the meantime.