hashicorp / terraform-plugin-testing

Module for testing Terraform providers
Mozilla Public License 2.0
44 stars 11 forks source link

Remove Terraform Installation Capabilities (TF_ACC_TERRAFORM_VERSION Environment Variable) #347

Open bflad opened 1 month ago

bflad commented 1 month ago

Description

The provider acceptance testing functionality (previously in terraform-plugin-sdk, now being updated in this separate Go module) has supported the ability to install Terraform from https://releases.hashicorp.com or build and install Terraform from Git source before running the testing logic that is dependent on Terraform being installed. This was offered for developer convenience and is enabled only when the TF_ACC_TERRAFORM_VERSION environment variable is set when a test is ran.

Since its introduction though, the testing world has changed. For example, GitHub Actions was introduced and bespoke actions such as github.com/hashicorp/setup-terraform can help automate Terraform installation in a much more flexible manner before invoking the actual provider testing.

Another oddity with Terraform installation via this Go module is that the installation can only be invoked for every helper/resource.Test()/helper/resource.UnitTest() invocation. No separate API was exported for performing that installation only once. For the testing environment, this is generally less than ideal.

Finally from a security-focused standpoint, the github.com/hashicorp/hc-install direct dependency which now handles this functionality causes the github.com/go-git/go-git/v5 indirect dependency to exist in any Go modules consuming this Go module. Given that Git and its libraries in general is a complex ecosystem and a consistent target for supply chain security and other security threats, many CVES affecting that ecosystem are generally not applicable to provider codebases, but their reports will show up in security scanning and cause unnecessary provider upgrade churn for practitioners looking or needing to ensure those reports are clean.

Proposal

Remove any code logic associated with installing Terraform. Dependencies such as github.com/go-git/go-git/v5 should no longer show up in this Go module's go.mod/go.sum.

For provider developers, this explicitly would mean that the TF_ACC_TERRAFORM_VERSION environment variable will have no effect after upgrading this Go module. Terraform will need to be separately installed and available on the PATH unless the TF_ACC_TERRAFORM_PATH override environment variable is set before individual tests are ran. Any upgrade documentation should call this out and potentially show how it can be done in common CI systems, such as GitHub Actions with setup-terraform.

References