Create shared environment file for ECS tasks

MackHalliday commented 2 months ago

User Story - Business Need

[x] Ticket is understood, and QA has been contacted (if the ticket has a QA label).

While working on #1711, we determined maintaining the environment in the task definition files would be easier if there was a share environment file per environment. The share environment file should be created in the same folder as the other json task files. The shared environment file would be shared among the api, celery, celery-beats, db-downgrade and db-upgrade task definition files.

Developer will have to determine all the unique environment per environment (dev, perf, staging, production) - and move them into a shared file.

ECS will need these integrated into the existing task definitions using a file with a .env extension, using the environmentFiles container defintion parmater

This user story depends on #895

User Story(ies)

As a VA Notify engineer, I want create a shared environment file for the task definition json files So that the task definitions json file is easier to maintain.

Additional Info and Resources

Spreadsheet reviewing all environment and secret keys in the task definition files. To see only the environment variables, filter by 'type'.
Environment variable files will be stored in environment-specific S3 buckets as specified in #895
AWS docs on environmentFiles
AWS docs on how to prepare the file

Acceptance Criteria

[x] Each environment has its own shared environment file
[x] env file is pushed to new S3 bucket (created in 895)
[x] ECS deploys without issue
[x] Unit and regression testing run as expected
[x] This work is added to the sprint review slide deck (key win bullet point and demo slide)

QA Considerations

Verify key functionality of application works as expected.

Potential Dependencies

MackHalliday commented 2 months ago

CC: @k-macmillan @npmartin-oddball

Hey! I realized I did not submit this ticket to review https://app.zenhub.com/workspaces/va-notify-620d21369d810a00146ed9c8/issues/gh/department-of-veterans-affairs/notification-api/1945 I apologize for the delay.

@Kyle MacMillan Re-reading Corey's ticket for infra and my own - I realized I'm unclear as to how exactly the new shared environment files will be uploaded to the new S3 buckets and then read from the S3 buckets.

Was hoping to talk to Corey to fill in the gaps but he has been out.

mjones-oddball commented 2 months ago

Hey team! Please add your planning poker estimate with Zenhub @cris-oddball @EvanParish @k-macmillan @kalbfled @mchlwellman

kbelikova-oddball commented 2 months ago

Please add your planning poker estimate with Zenhub @coreycarvalho

coreycarvalho commented 3 weeks ago

Buckets have all been created and are ready for environment files.

EvanParish commented 3 weeks ago

I created the shared env files for each environment. Tomorrow I'll work on the actions to get the files uploaded to S3 when deploying the app.

EvanParish commented 3 weeks ago

I was reading an article that I now realize may have been over-complicating the situation for our use-case. I'm going to try simply using an AWS CLI command to upload the file. I just have some questions about the sensitivity of the ARNs that need to be added to the repo. There are others that exist already, but just want to double check before committing that information.

EvanParish commented 2 weeks ago

I deployed this work this morning and everything seems to be working correctly in dev!

cris-oddball commented 2 weeks ago

PR approved and merged, sending to Perf and holding there.

cris-oddball commented 2 weeks ago

Looks good, but should probably send and hold on staging where there is more traffic and take a look at Datadog to make sure everything is good.

cris-oddball commented 2 weeks ago

Looking good.

department-of-veterans-affairs / notification-api