search

nsmader / acs-constructicon

This repository represents a set of tools for crowd-sourcing generation of secondary measures constructed from the American Community Survey (ACS).
0 stars 1 forks source link

Update pull of field names through get-acs-meta-data.R script #3

Open imorey opened 8 years ago

nsmader commented 8 years ago

Currently, the get-acs-meta-data.R pulls--and pulls apart--meta-data on field names, but only for 2013 ACS data, and even that is based on an external document. We should update this to:

  1. Identify all tables in all release years of the ACS, ideally based on the Census API itself (rather than a separate file)
  2. Loop through all of those table-by-year combinations to harvest field names
  3. Feed that into the current code to parse out all unique field components (e.g. break "male :: civilian :: in labor force :: employed" into c("male", "civilian", "in labor force", "employed"), and compile those across all table fields)
  4. Merge to that list the current field components that we've coded
dylan-stark commented 8 years ago

I've got a start at this in the variables.R script over on the packaging branch. Step 2 is implemented in the for loop over raw_files. Steps 3 and 4 are (mostly) handled after that. The code drops any variables that it cannot fully code because one or more components do not have an abbreviated version defined in data-raw/codes.csv (a copy of one of the original code files).

Step one isn't automated through the Census API, it is just hardcoded in the raw_files list. It could be extended if there's a good source for that data provided through the API.

Note this process is done when building the package and the results are stored within the package, so no user would have to run this script.