cfpb / api

Documentation to support upcoming data platform API and data sets
http://cfpb.github.io/api/hmda/
31 stars 73 forks source link

Add 2013 MSAMD mappings #46

Closed m3brown closed 9 years ago

m3brown commented 9 years ago

2013 records have been added to the msasm.csv file, and the file was re-sorted using the following command:

sort -n --field-separator=',' -k2 -k1 msamd.csv

Also included is a script to ingest data from a CSV or XLSX in the correct format for the msamd.csv file. For example, to add 2014 records:

# output the content to stdout for validation
./generate_msamd_csv.sh -y 2014 input-file.csv

# backup the original end file and apply new records
./generate_msamd_csh.sh -y 2014 -a code_sheets/msamd.csv input-file.csv

TODO: The script does not handle a couple edge cases with upper/lowercasing:

kave commented 9 years ago

@m3brown in your comment "backup the original end file and apply new records" where does it back up the file too? Does this command just modify the existing msamd.csv file with the new year?

CFPBadmin commented 9 years ago

It backs up the file to filename.timestamp. It deletes and records that exist for the year you specified, appends the new records to the bottom, then sorts the file to integrate the new year into the original formatting. I can improved the documentation for this.

kave commented 9 years ago

@m3brown you've resolved all my questions, approved!