AFT upgrade fails when using TFC without setting AWS_DEFAULT_REGION envvar

[Tmills89]

Terraform Version & Prov:

AFT Version: 1.3.3

Terraform Version & Provider Versions Please provide the outputs of terraform version and terraform providers from within your AFT environment

terraform version

Terraform v0.15.5

terraform providers

aws_provider ">= 3.72, < 4.0.0"

Bug Description When updating AWS Control Tower Terraform workspace from 1.3.3 to 1.6.6 the following errors: Error: error configuring Terraform AWS Provider: error validating provider credentials: error calling sts:GetCallerIdentity: operation error STS: GetCallerIdentity, failed to resolve service endpoint, an AWS region is required, but was not found

Error: Invalid provider configuration

To Reproduce Steps to reproduce the behavior:

1. Create branch
2. pull latest commits (1.6.6)
3. merge branch into main
4. run plan and apply on terraform control tower workspace

Expected behavior Plan and apply completes with update to 1.6.6

Related Logs

2022-10-17T20:11:02.443Z [TRACE] buildProviderConfig for provider["registry.terraform.io/hashicorp/local"]: no configuration at all

2022-10-17T20:11:06.481Z [ERROR] vertex "provider[\"registry.terraform.io/hashicorp/aws\"]" error: error configuring Terraform AWS Provider: error validating provider credentials: error calling sts:GetCallerIdentity: operation error STS: GetCallerIdentity, failed to resolve service endpoint, an AWS region is required, but was not found 2022-10-17T20:11:06.481Z [ERROR] vertex "provider[\"registry.terraform.io/hashicorp/aws\"]" error: Invalid provider configuration

Additional context Error: error configuring Terraform AWS Provider: error validating provider credentials: error calling sts:GetCallerIdentity: operation error STS: GetCallerIdentity, failed to resolve service endpoint, an AWS region is required, but was not found with provider["registry.terraform.io/hashicorp/aws"] on line 0:

Error: Invalid provider configuration Provider "registry.terraform.io/hashicorp/aws" requires explicit configuration. Add a provider block to the root module and configure the provider's required arguments as described in the provider documentation.

[hanafya]

Hey @Tmills89, Can you confirm that you have upgraded your providers? We have noticed this issue before with older providers. If you are still running into this issue after upgrading your providers I would recommend reaching out to AWS Premium Support

[Tmills89]

I ran terraform init on the workspace and still ran into the same error. I also confirmed that the provider is within the workspace meets the requirements

Providers required by configuration: . ├── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0, < 5.0.0 ├── provider[registry.terraform.io/hashicorp/local] ├── module.aft_code_repositories │ ├── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0 │ └── provider[registry.terraform.io/hashicorp/local] ├── module.aft_account_request_framework │ ├── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0 │ └── provider[registry.terraform.io/hashicorp/time] ├── module.aft_backend │ └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0 ├── module.packaging │ └── provider[registry.terraform.io/hashicorp/archive] ├── module.aft_ssm_parameters │ ├── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0 │ └── provider[registry.terraform.io/hashicorp/random] ├── module.aft_account_provisioning_framework │ └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0 ├── module.aft_customizations │ ├── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0 │ └── provider[registry.terraform.io/hashicorp/local] ├── module.aft_feature_options │ └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0 ├── module.aft_iam_roles │ ├── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0 │ ├── module.ct_management_exec_role │ └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0 │ ├── module.ct_management_service_role │ └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0 │ ├── module.log_archive_exec_role │ └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0 │ ├── module.log_archive_service_role │ └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0 │ ├── module.aft_exec_role │ └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0 │ ├── module.aft_service_role │ └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0 │ ├── module.audit_exec_role │ └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0 │ └── module.audit_service_role │ └── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0 └── module.aft_lambda_layer ├── provider[registry.terraform.io/hashicorp/aws] >= 4.9.0 ├── provider[registry.terraform.io/hashicorp/random] └── provider[registry.terraform.io/hashicorp/local]

Providers required by state:

provider[registry.terraform.io/hashicorp/random]

provider[registry.terraform.io/hashicorp/local]

provider[registry.terraform.io/hashicorp/time]

provider[registry.terraform.io/hashicorp/archive]

provider[registry.terraform.io/hashicorp/aws]

[docspy]

I had the same error: It turned out that I missed setting a default region. Adding the export AWS_DEFAULT_REGION=<region> did it. I also updated in steps 1.3.3 -> 1.3.6 -> 1.4.2 -> 1.5.2 -> 1.6.6 to make sure I did not miss something inbetween. Though I am not sure if it is realy necessary to do so.

[smokentar]

232 for reference

[abhishek-sorenson]

I had the same error: It turned out that I missed setting a default region. Adding the export AWS_DEFAULT_REGION=<region> did it. I also updated in steps 1.3.3 -> 1.3.6 -> 1.4.2 -> 1.5.2 -> 1.6.6 to make sure I did not miss something inbetween. Though I am not sure if it is realy necessary to do so.

Hi @docspy, yup that did end up fixing the issue. We're running Terraform Cloud and we set that variable in the workspace as an environment variable. I still want AWS to either provide some sort of fix for this or provide some guidance in their docs somewhere to have the AWS_DEFAULT_REGION variable in place to avoid this error. @smokentar has experienced the same problem before and sounds like we're both using Terraform Cloud. For AWS to recreate this issue, they'll need an environment in which that environment variable is not set.