Store Income Limits CSVs in S3 bucket

[wesrowe]

Description

User story

AS A Veteran I WANT new CSVs of Income Limits data to be stored in an S3 bucket SO THAT it can be loaded into my Income Limits experience in an automated process.

Engineering notes / background

• convert Excel files to CSV (save as...)
• ID (and other "integer" columns are comma-separated thousands formatted; need to set correct format before exporting to .csv in Excel file
• Platform CMS folks (Nate, Tim, Eric) have AWS access & volunteered to help push to S3
• Public Websites team should get set up for AWS access, to manage this process ourselves moving forward

Quality / testing notes

Acceptance criteria

• [x] AWS access requests are understood and created
• [x] All five files for the migrations created in #13319 are stored in an S3 bucket (as CSV) accessible to the cron job that will load them into vets-api environments
• [x] Document the data path to S3 in a new team "folder" - va.gov-team/tree/master/products/income-limits-app/data

[dsasser]

AWS Access

AWS access is requested by creating a new Github issue in the va.gov-teams repo. There is an AWS Access Request template. The source of this information is in the Platform documentation.

My AWS access request.

Data Files

Thanks to @olivereri we now have a new S3 bucket sitewide-public-websites-income-limits-data (named such that it is easy for Platform to identify the owner and intent of the bucket).

The files are publicly available at the following URLs: https://sitewide-public-websites-income-limits-data.s3-us-gov-west-1.amazonaws.com/std_zipcode.csv https://sitewide-public-websites-income-limits-data.s3-us-gov-west-1.amazonaws.com/std_state.csv https://sitewide-public-websites-income-limits-data.s3-us-gov-west-1.amazonaws.com/std_incomethreshold.csv https://sitewide-public-websites-income-limits-data.s3-us-gov-west-1.amazonaws.com/std_gmtthresholds.csv https://sitewide-public-websites-income-limits-data.s3-us-gov-west-1.amazonaws.com/std_county.csv

This has been documented in the Teams repo here.

[dsasser]

@jtmst you should now be unblocked for #13320.

[wesrowe]

@dsasser, on the CSV conversion instructions I see your step for creating new worksheets to copy the data into, but I'm missing the steps where one actually exports the CSV from Excel. I think I could figure it out, but you might as well make it explicit.

Edit: Otherwise, I should say that the documents seem very clear. Thank you :)

[dsasser]

@dsasser, on the CSV conversion instructions I see your step for creating new worksheets to copy the data into, but I'm missing the steps where one actually exports the CSV from Excel. I think I could figure it out, but you might as well make it explicit.

Added, thanks for calling that out.