Next steps coding

[rebeccajohnson88]

Some cleanup of rowbinding code @rebeccajohnson88

• [x] Relative rather than absolute path names

## relative path name -- should work regardless of user if we have same directory structure (can sync at the meeting)
OUTPUT_DIR = "../../qss20_finalproj_rawdata/summerwork/raw/"

## absolute pathname has my username in it, eg
/Users/rebeccajohnson/Dropbox/qss20_s21_proj/code

• [x] Cam's code in issue fully subsumed in script 02?

Cleaning of job addresses/geocoding:- maybe @helenma0223234 ?

• [x] Inspect addresses in h2a_combined_2014-2021 and clean as needed
• [x] Copy over this script to the main code: https://github.com/rebeccajohnson88/qss20_s21_proj/blob/main/code/qss20_groupcode/geocoding_acs/01_geocode_mergewithACS_plot_studentversion.ipynb and move the API key to a .yaml file called creds.yaml (can put in dropbox; not github)
• [x] Geocode a sample of addresses (eg maybe 100) and check runtime
• [ ] Geocode all addresses using a .py --- may want to simplify the job clearance data to unique addresses and create a crosswalk between those and case numbers (so if two different job orders have the same address, that address just gets geocoded once and then the lat/long are merged back to both job orders)

Check ACS codebook and pull additional ACS characteristics for all census tracts: - maybe @Grant-98 and @eunice30718

• [ ] This pulls a few ACS numerators/denominators - https://github.com/rebeccajohnson88/qss20_s21_proj/blob/main/code/qss20_groupcode/geocoding_acs/01_geocode_mergewithACS_plot_studentversion.ipynb
• [ ] @rebeccajohnson88 will share ACS codebook and identify what year to pull
• [ ] Identifying a larger number of predictors, marking them as "yes" on a sheet, and pulling for all tracts / states

Fuzzy matching between job orders data and violations data- maybe @camguage

• [ ] @rebeccajohnson88 will share fastlink script- https://cran.r-project.org/web/packages/fastLink/index.html
• [ ] Group did fuzzy matching in python; loads WHD investigations data - https://github.com/rebeccajohnson88/qss20_s21_proj/blob/main/code/qss20_groupcode/predict_viol/A1_fuzzy_matching.py switching to R; would still fuzzy match on employer name and either address or city and maybe block on state; would do in .R script and start out with smaller sample given runtime

[rebeccajohnson88]

Running threads to discuss on Monday:

• Fuzzy matching: @camguage closing out the job clearances -> investigation linkages--- discuss ways to construct outcome variable for predictive models

• Geocoding: @helenma0223234 issues with running .py version- Qs from me on slack about IDE

• ACS: @rebeccajohnson88 looking for unemployment vars; @eunice30718 and @Grant-98 closing out

• TRLA intake data: @rebeccajohnson88 close out data cleaning and formulate script with questions for lizzie --- hierarchical coding of adverse party; then pass to cam for fuzzy matching based on adverse party