Terragrunt to read account IDs & co. from the tfvars file

[P0NDER0SA]

Description

As a Developer, I need to be able to use Terraform variables in code such as the AWS account ID, so that I can achieve secure and maintainable code.

WHY are we building?

Code security and maintainability.

WHAT are we building?

• Variables in code from Terraform can be read from Terragrunt, such as AWS Account IDs.
• Proof of concept with having Terragrunt reading from the tfvars files, consolidating the account ID in one of these. This should be done with minimal effort. The #109 task would tackle the next big steps.
• We need to have Terragrunt reading the tfvars.

VALUE created by our solution

Secure and consistent/debuggable code.

QA Steps

• [x] Search across entire TF repository for hardcoded AWS account ids
• [x] All Plans and Applies are working against Staging and Production
• [x] Verify that the account ID is consolidated within one tfvars file, read from TG and demonstrated as a proof of concept for #109.

Acceptance Criteria

• [ ] Scenario: Tokenizing TF variables such as AWS Account IDs in Terraform Code
  • Given the issue is about tokenizing AWS Account IDs in Terraform code
  • Account IDs are stored in a tfvars file.
  • No AWS Account Ids are visible in code repo

[P0NDER0SA]

Identified a list of all of the AWS account ids we have in our Terraform code. Investigated the best way to set up variables for them -- decided that we should focus on the CDS internal ids (as many of the other external ones are public anyways).

In order to use TFVars variables for these we really need to adjust our code to allow Terragrunt to use our TFVars file (which is scoped to Terraform for some silly reason).

Discovered a newer function that could help us do just that-- need to compare this with Bens other env enhancements and decide how to proceed. Also if we use this, we will have to update our actions to use a newer version of the Terragrunt Actions.

[P0NDER0SA]

https://github.com/cds-snc/notification-terraform/pull/1498

[P0NDER0SA]

Started on the implementation of the read_tfvars_function for dev and reviewing Bens amalgamated tfvars file for dev

[P0NDER0SA]

I have a working model for this on Dev, but i need to get it workign on Staging before I can merge it.

[P0NDER0SA]

Blocked by an issue with GHA that i can't reproduce locally. Will bug Ben when he's available. Updated the code to use variables (on dev and staging and sandbox environments).

Need to fix the plans for Staging before devising a way to keep production working as it does until i can get staging 100%.

https://github.com/cds-snc/notification-terraform/pull/1516

[P0NDER0SA]

Ben and I want to debug this more and find out why the planning is failing without any errors/warnings today

[ben851]

For dev TF Plan github action, I just switched the role to notification-terraform-apply so that the DNS management works.

[P0NDER0SA]

amalgamated all configuraable values in staging into our tfvars file (this was a big job!) and created all variable declarations in our top level Terragrunt hcl file. Currently working on removing redundant variable declarations in downstream variable files for staging. We very much underestimated this one!

[P0NDER0SA]

all variables have been pulled together for dev and staging and are in 1pass tfvars files. they are extensive. Plans are working against dev and staging -- we need to verify the plans to make sure they aren't doing anythign dangerous (the new sensitive values are causing it to show a good many "changes" that might not really be changes). Making very good progresss.

[sastels]

Working through applies and debugging. Some issues arose on dev (VPN randomly breaking).

[sastels]

Notes don't seem to be versioned in 1Password - should investigating.

Update: It's versioned but you need to go through web portal to get to the history. Pond will add this to Tips and Tricks

[sastels]

VPN working again on dev :tada: Next:

• apply new code to dev and verify no strange changes
• look at staging.

[sastels]

Gonna look at the prod plan first.

[jimleroyer]

We got dev and staging ready to go with the changes for this PR. We'll wait on Pond before resuming on that task. Pond was working on the production workflows last week.

[jimleroyer]

Today

• we want to apply the plan in staging.
• run the plan against production.
• find discrepancies between GitHub passwords and 1password passwords.

[P0NDER0SA]

new PR has been created with minimal commits! https://github.com/cds-snc/notification-terraform/pull/1562

[jimleroyer]

• Let's create a revert PR for that giant PR in case that something happens and we need a quick rollback. The changes in staging will be sitting for the next few days.
• We need all the tfvars for the production environment.

[P0NDER0SA]

Merged! It's deployed on staging.

One small issue with Heartbeat - just an out of date config value for the API key. Fixed!

We will re-create a new api key for production before we release to nip this in the bud.

TFVars for prod have been reviewed. One more pass on this today and a Plan against production before release!

[P0NDER0SA]

released to production this morning! Looks good so far :)

We might look at making some small changes to move "configurations" back to the code base for traceability and leaving the secrets in 1pass. They will still be fairly centralized and easy to track

[P0NDER0SA]

Looking good in Production!

[sastels]

Steve will fix account ids in:

• [x] check_rds_cluster_update.yml (can pull from github secrets)
• [x] terragrunt_destroy_environment.yml (can pull from github secrets)
• [x] file_scanning.tf (add to 1Password)
• [x] ec2.tf (add to 1Password)
• [x] sentinel.tf (add to 1Password)
• [x] ecr_user.tf (add to 1Password)
• [x] aws_integration.tf (new relic) (add to 1Password)

[sastels]

PR to Move out a couple account ids

[sastels]

I'll do another check today to double check that we got everything.

[sastels]

dang it, I missed one. PR here

[sastels]

All good now!