search

walkerke / tidycensus

Load US Census boundary and attribute data as 'tidyverse' and 'sf'-ready data frames in R
https://walker-data.com/tidycensus
Other
639 stars 100 forks source link

ACS5 fails to download for some years(2011 and 2012) but not others. #52

Closed RyanWalch closed 6 years ago

RyanWalch commented 6 years ago

I get an error running this code

state_income <- get_acs(geography = 'tract',
                          table = c("B19301"),
                          state = 'AZ',
                          year = 2012,
                          survey = 'acs5',
                          geometry = TRUE,
                          cache_table = TRUE)

The error

''' Please note: get_acs() now defaults to a year or endyear of 2016. Downloading feature geometry from the Census website. To cache shapefiles for use in future sessions, set options(tigris_use_cache = TRUE). Using FIPS code '04' for state 'AZ' trying URL 'http://www2.census.gov/geo/tiger/GENZ2012/cb_2012_04_tract_500k.zip' Error in download.file(url, tiger_file, mode = "wb") : cannot open URL 'http://www2.census.gov/geo/tiger/GENZ2012/cb_2012_04_tract_500k.zip' In addition: Warning message: In download.file(url, tiger_file, mode = "wb") : cannot open URL 'http://www2.census.gov/geo/tiger/GENZ2012/cb_2012_04_tract_500k.zip': HTTP status was '404 Not Found' '''

It doesn't appear to be an internet connection issue, my internet is working fine otherwise and I can download the 2010,2013,2014,2015 data no problem. I can also download the 2012 data through the ACS package with no problems.

walkerke commented 6 years ago

Thanks, this will be a quick fix. The issue is with the geometry for those years, so if you set geometry to FALSE it'll work for now. I'll get this taken care of shortly.

walkerke commented 6 years ago

This is fixed in https://github.com/walkerke/tidycensus/commit/02e3b116868546f00ed03c7e0d53af54eda76d99. tidycensus uses the cartographic boundary files available from tigris by default, but can fetch the regular TIGER/Line files with the keyword argument cb = FALSE. However, cartographic boundary files do not exist for 2011 and 2012, so tidycensus now uses the TIGER/Line files by default for those years.

philiporlando commented 5 years ago

I'm getting a similar error for Detroit's metro area. Setting cb = F returns geometries without error, but I'm confused as to why we're still seeing this error for the default year?

dma <- get_acs(
  geography = 'block group'
  ,state = 'MI'
  ,county = c('Oakland', 'Macomb', 'Wayne')
  ,variables = c('B22010_001E','B22010_002E')
  ,geometry = T
  ,output = 'wide'
  ,cb = T # throws an error if TRUE!
)
Getting data from the 2013-2017 5-year ACS
Downloading feature geometry from the Census website.  To cache shapefiles for use in future sessions, set `options(tigris_use_cache = TRUE)`.
Getting data from the 2013-2017 5-year ACS
Downloading feature geometry from the Census website.  To cache shapefiles for use in future sessions, set `options(tigris_use_cache = TRUE)`.
Using FIPS code '26' for state 'MI'
Using FIPS code '125' for 'Oakland County'
Cannot open layer cb_2017_26_bg_500k
Error in CPL_read_ogr(dsn, layer, query, as.character(options), quiet,  : 
  Opening layer failed.
In addition: Warning message:
In unzip(file_loc, exdir = tmp) : error 1 in extracting from zip file

Setting cb = F is working for me.

dma <- get_acs(
  geography = 'block group'
  ,state = 'MI'
  ,county = c('Oakland', 'Macomb', 'Wayne')
  ,variables = c('B22010_001E','B22010_002E')
  ,geometry = T
  ,output = 'wide'
  ,cb = F # throws an error if TRUE!
)
Getting data from the 2013-2017 5-year ACS
Downloading feature geometry from the Census website.  To cache shapefiles for use in future sessions, set `options(tigris_use_cache = TRUE)`.
Getting data from the 2013-2017 5-year ACS
Downloading feature geometry from the Census website.  To cache shapefiles for use in future sessions, set `options(tigris_use_cache = TRUE)`.
Using FIPS code '26' for state 'MI'
Using FIPS code '125' for 'Oakland County'
Getting data from the 2013-2017 5-year ACS
Downloading feature geometry from the Census website.  To cache shapefiles for use in future sessions, set `options(tigris_use_cache = TRUE)`.
Using FIPS code '26' for state 'MI'
Using FIPS code '099' for 'Macomb County'
Getting data from the 2013-2017 5-year ACS
Downloading feature geometry from the Census website.  To cache shapefiles for use in future sessions, set `options(tigris_use_cache = TRUE)`.
Using FIPS code '26' for state 'MI'
Using FIPS code '163' for 'Wayne County'
walkerke commented 5 years ago

@philiporlando thanks for bringing this up. I've also been experiencing similar behavior intermittently, though I just ran your problem code successfully.

Specifically, connections to the Census shapefiles via httr GET requests (which is used deep under the hood here) have been known to occasionally fail. I've generally noticed these errors when R is used in a server environment - both on Windows and Linux. I've pinged Census about this, but the error has been tough to isolate as they've had to dig through server logs and the problem generally goes away.

Give it another try and hopefully it'll work as it did for me. If not I'll touch base with Census again.

janemumei commented 5 years ago

I've been getting this error as well, even on the latest version of R (3.5.1), the ACS 5-year published in 2010, 2011, 2012, 2013, and 2014.

> get_acs(geography = "place",
          state     = "NY",
          variables = c("B01003_001E"),
          year = 2013,
          survey = "acs5",
          geometry = FALSE)
Getting data from the 2009-2013 5-year ACS
Using FIPS code '36' for state 'NY'
Client error Not Found Client error: (404) Not Found
Error in as.character(x) : 
  cannot coerce type 'closure' to vector of type 'character'
mfherman commented 5 years ago

Hi @jonoyuan, I just ran this code succesfully. It could be that Census API was experiencing problems or your internet connectivity was spotty when you tried to run it.

library(tidycensus)

get_acs(geography = "place",
        state     = "NY",
        variables = c("B01003_001E"),
        year = 2013,
        survey = "acs5",
        geometry = FALSE)
#> Getting data from the 2009-2013 5-year ACS
#> # A tibble: 1,190 x 5
#>    GEOID   NAME                       variable   estimate   moe
#>    <chr>   <chr>                      <chr>         <dbl> <dbl>
#>  1 3600155 Accord CDP, New York       B01003_001      873   487
#>  2 3600199 Adams village, New York    B01003_001     1769   277
#>  3 3600232 Adams Center CDP, New York B01003_001     2286   383
#>  4 3600276 Addison village, New York  B01003_001     1840   154
#>  5 3600342 Afton village, New York    B01003_001     1048   184
#>  6 3600408 Airmont village, New York  B01003_001     8692    34
#>  7 3600441 Akron village, New York    B01003_001     2862    18
#>  8 3601000 Albany city, New York      B01003_001    98142    70
#>  9 3601011 Albertson CDP, New York    B01003_001     5260   481
#> 10 3601033 Albion village, New York   B01003_001     5595   285
#> # ... with 1,180 more rows

Created on 2019-04-24 by the reprex package (v0.2.1)

jefunes commented 4 years ago

I am trying to retrieve some data from ACS but I keep getting error just for specific tables, see below life_phy_ssc_var <- c("C24010_010"," C24010_046") life_phy_ssc_DC <- get_acs(geography = "County", variables = life_phy_ssc_var, state="DC", year=2018) Getting data from the 2014-2018 5-year ACS Using FIPS code '11' for state 'DC' No encoding supplied: defaulting to UTF-8. Error: Your API call has errors. The API message returned is .

jefunes commented 4 years ago

the for census tract life_phy_ssc_var <- c("C24010_010"," C24010_046") life_phy_ssc <- get_acs(geography = "tract", variables = life_phy_ssc_var, state="DC", geometry = TRUE, year=2018)

Getting data from the 2014-2018 5-year ACS Downloading feature geometry from the Census website. To cache shapefiles for use in future sessions, set options(tigris_use_cache = TRUE). Using FIPS code '11' for state 'DC' No encoding supplied: defaulting to UTF-8. Error: Your API call has errors. The API message returned is .

walkerke commented 4 years ago

@jefunes your code works fine for me once the space is removed prior to the second variable (you're sending the space to the API in the variable name which it is not understanding).

That said I was unable to reproduce your first error. Could you make sure all of your packages are updated? If that does not solve it, could you paste the results of your sessionInfo() here?

jefunes commented 4 years ago

space made a difference

jefunes commented 4 years ago

thanks!