new 'hashicorp/aws' version (> v5.0.0) cause the incompatible issue for version that defined inside [../sources/aft-customizations-common/templates/customizations_pipeline/versions.tf]

[UlaJiang]

Bug Description 'hashicorp/aws' v5.0.0 was released at the end of May [https://github.com/hashicorp/terraform-provider-aws/releases], which cause the incompatible issue for version that defined inside [../sources/aft-customizations-common/templates/customizations_pipeline/versions.tf]

e.g. https://github.com/aws-ia/terraform-aws-control_tower_account_factory/blob/1.4.2/sources/aft-customizations-common/templates/customizations_pipeline/versions.tf

terraform {
  required_version = ">= 0.15.1"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.9.0"
    }
  }
}

So when 'terraform init', it will trying to Install the newest 'hashicorp/aws' version, log from the codebuild log:

[Container] 2023/06/06 16:29:50 Running command terraform init

Initializing the backend...

Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Finding hashicorp/aws versions matching ">= 4.9.0"...
- Installing hashicorp/aws v5.1.0...
- Installed hashicorp/aws v5.1.0 (signed by HashiCorp)

Which cause the below error whilen doing 'terraform apply'

[Container] 2023/06/06 16:29:54 Running command terraform apply -var="account_id=$VENDED_ACCOUNT_ID" --auto-approve
╷
│ Error: Unsupported argument
│ 
│   on aft-providers.tf line 7, in provider "aws":
│    7:   shared_credentials_file = "~/.aws/credentials"
│ 
│ An argument named "shared_credentials_file" is not expected here. Did you
│ mean "shared_credentials_files"?

To temperary fix the issue, we change the 'versions.tf' file after in our local repo which is from the [../sources/aft-customizations-common/templates/customizations_pipeline/versions.tf] as below (e.g. https://github.com/aws-ia/terraform-aws-control_tower_account_factory/blob/1.4.2/sources/aft-customizations-common/templates/customizations_pipeline/versions.tf)

terraform {
  required_version = ">= 0.15.1"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.9.0 <5.0.0"     <<<<<----- HERE
    }
  }
}

Question

1. Can we confirm that this is a potential bug this will be take care in the future release?
2. I noticed that for 'versions.tf' inside the [../sources/aft-customizations-common/templates/customizations_pipeline/versions.tf], beside the main branch, all the Tags version (e.g. 1.4.2) are all defined version = ">= 4.x.0" without mentioned "<5.0.0", so may I also know if the fixed of 'versions.tf' will also take care previous Tags version's 'versions.tf' please?

Thanks

[balltrev]

Hey @UlaJiang thanks for reporting this. Can you provide the AFT version you have deployed?

I've looked at the latest source code for that provider file and I see that it properly lists the argument as shared_credentials_files

As for the < 5.0.0 version restraint, I've gone ahead and created a backlog to address that with the team.

[UlaJiang]

Hi @balltrev Thanks for your quick reply.

The Tag version that thrown the error is 1.4.2, from the provider file of this version I can see that shared_credentials_file instead of shared_credentials_files, which got a s missing in the end. From the AWS Provider I can also confirm that the environment variables is shared_credentials_files (with s).

One interesting bahavior we observe is after we apply the version = ">= 4.9.0 <5.0.0" in versions.tf, and reapply the Tag 1.4.2 buildspecs yaml file, we are able to avoid the above error An argument named "shared_credentials_file" is not expected here. Can you please help us understand better on this behavvior please? Thanks.

[balltrev]

You've called out the reason for failure in the v1.4.2 deployment; due to the lack of ,< 5.0.0 in the version constraint, AFT at runtime updates to a version of the hashicorp/aws provider that does not support shared_credentials_file argument. Adding the constraint fixes this issue.

Our recommended solution here is to upgrade to the latest version of AFT, as it resolves several issues, and will not require you to fork the package to modify version.tf files.

[stumins]

Resolving due to inactivity; please open a new issue if you continue to experience this issue on the latest version of AFT.