Closed hobuobi closed 3 years ago
Below are the relevant details for each step. They came out longer than I expected... Let me know if you have questions!
cat $FILE.json | sort | awk -F ',"date_published":"[0-9]{4}-[0-9]{2}-[0-9]{2}"' '{print $1,$2}' > $FILE.new.json
src/data/
and run the app locally. We don't have a state switcher so you have to update the state in src/index.js
. It is a bit slow so I just do it to spot check anything I was worried about or that CSG emphasized.July data collection is just about done! A few states had new data sources or metrics and will require YAML changes:
A few states have not been completed yet, but we should still go ahead and proceed with ingest and deal with these next week:
All of this has been ingested. While I was at it I pulled the data for Delaware, finished off TX (I think it had already been done, but there were three prison population points that were missing), and ingested what had been backfilled so far for NC.
I think the only thing that leaves is ingesting any further backfill that is done for NC admissions.
Amaaaaazing! I think it's probably still valuable for us to send over what we've got. :) They also requested getting a CSV like the one we made for them last time with all the data (that went over to Jess); if that's quick enough to re-run and include when we send the new data file, great! If not, we can follow-up whenever it's convenient for us this week (and I can create a new ticket if needed).
I sent all of this off to CSG. Should we open another ticket for the NC backfill? Or are we dropping that from scope?
July data collection consists of our data entry contractors entering the data from state reports, dashboards, etc. into the "XX_Data" sheets found in these folders: https://drive.google.com/drive/folders/1hhyCN5O5HBD3rl2BncHhQ1GzwV7x41Cz
Once that process is complete, we simply need to:
Note that this process does not require any communication with the contractors; any necessary communication there will be done by @hobuobi.