Updating Default VPC Terraform module version

[endzyme]

• Adjusted default version to be v3.0.0
• Adjusted to remove the tf_var aws_secret/access_key settings (they aren't needed anymore)
• Adjusted to remove those tf_vars from secrets.yml default script

The module doesn't need to have settings for provider since they are invoked by the main instance (inherited). I also removed these keys from secrets and other areas as they are not used outside of this module use case.

NOTE: I have not made a migration for the following reasons:

1. There is no way to tell if someone has started using TF_VAR_aws_access_key or TF_VAR_aws_secret_key since the project was initialized.
2. I would not feel comfortable editing secrets.yml with a migration to remove the settings as well as adjusting the version of the terraform to be auto-bumped to v3.0.0. This should be handled with auditomation.

[endzyme]

Change merged https://github.com/reactiveops/terraform-vpc/pull/

[ejether]

In my opinion, TF_VAR_aws* should be removed with a migration. The version of the module should be in auditomation but perhaps it should also get bumped by them migration? I could go either way on. If we bump the version of the module here, then we escape the problem with having the two projects being tightly coupled?

Are you worried that the change in Terraform module will have unexpected consequences and shold not be updated automatically?

Generally speaking, migrations are safe. Nothing irrevocable can be done with a migration. The "worst case" is needing to revert that specific commit.

[endzyme]

Here is the workflow I am envisioning, and supporting reasons to my decision to not include a migation.

1. We have a task to upgrade the pentagon repo, or we just notice that there's a new version
2. We upgrade it and commit it without running terraform (since we rarely run terraform unless it needs a change)
3. Later in life (usually weeks or months later) someone runs terraform to find it doesn't work anymore, it becomes difficult to discern what caused the breakage and probably what the person will do is reintroduce the TFVAR to get around the problem

I can also see people (myself included) making the migration and then not realizing you need to push the new secrets to 1password - just causing confusion (since it's not listed in the git commit delta)

I strongly think we should not add extra work that, in my opinion, will lead to further confusion with migrations.

[ejether]

Could this be prevented by running Terraform more often? Or by adding notes/documentation to the migration? Or informing, up front, those running the migrations that Terraform needs to be run?

It seems to me you are displacing the 'extra work' from here to there and punting it down the road. The change still needs to be made does it not?

[endzyme]

Maybe this is something you could outline in a document and put into ROTime since it's outside of the scope for my specific change for SRE work and I honestly don't have the time to tackle the larger problem of why people don't run terraform more often.

Thanks for the feedback.

[davekonopka]

I'm coming in a little late here but I see a lot of value adding this as a migration. The previous VPC module code breaks terraform import. Anytime we need to import something we then need to either know to upgrade the VPC module or temporarily remove TF code. Fixing that seems worth including a no-op TF change by migration.

Side note, I would expect anyone running migrations to test terraform plan before committing code. If they don't that definitely sets up potential for an ugly situation down the road.

@endzyme If this isn't somewhere you can spend time I'd be willing to spend some RO time adding the migration. I would test it on one of our pod repos to make sure it's a clean no-op.

[ejether]

Reopening because I see value in the work and conversation

[endzyme]

@davekonopka Thank you for taking this on. Please don't hesitate to reach out with questions or context on what's in play currently. I've also written a migration with pentagon in the past so feel free to ask away on that too.

Thanks for moving this forward!

[ejether]

@davekonopka When you're ready to create one, let me know and I can walk you through it as needed. You should also be able to reference the existing ones to gooed effect.

[CLAassistant]

All committers have signed the CLA.

[davekonopka]

@ejether @endzyme I merged in master to get the 2.5.0 changes, bumped to 2.6.0, and added a migration.

The migration:

• Bumps the VPC TF module to 3.0.0
• Removes AWS key and secret variables from module resource
• Removes AWS key and secret TF variables from secrets.yml and outputs a warn message to commit the changed files to 1Password.

I tested this out on ***** repo. After the migration I got the changes I expected, ran terraform init and then terraform plan showed no changes.

[ejether]

I checked a few other repos. All worked for me.