Cannot get s3:: modules with AWS SSO

endzyme commented 3 months ago

Terraform Version

Terraform v1.7.4
on linux_amd64

Terraform Configuration Files

module "cluster" {
  source = "s3::https://s3.amazonaws.com/redacted-s3-bucket-name/some-module-name.tar.gz"

  ...
}

module "workers" {
  source = "s3::https://s3.amazonaws.com/redacted-s3-bucket-name/some-module-name.tar.gz"

  ...
}

Debug Output

$> TF_LOG=debug tf get -update

2024-03-04T21:34:57.178-0700 [INFO]  Terraform version: 1.7.4
2024-03-04T21:34:57.178-0700 [DEBUG] using github.com/hashicorp/go-tfe v1.41.0
2024-03-04T21:34:57.178-0700 [DEBUG] using github.com/hashicorp/hcl/v2 v2.19.1
2024-03-04T21:34:57.178-0700 [DEBUG] using github.com/hashicorp/terraform-svchost v0.1.1
2024-03-04T21:34:57.178-0700 [DEBUG] using github.com/zclconf/go-cty v1.14.1
2024-03-04T21:34:57.178-0700 [INFO]  Go runtime version: go1.21.5
2024-03-04T21:34:57.178-0700 [INFO]  CLI args: []string{"/User/someone/path/.tfenv/versions/1.7.4/terraform", "get", "-update"}
2024-03-04T21:34:57.178-0700 [DEBUG] Attempting to open CLI config file: /User/someone/path/.terraformrc
2024-03-04T21:34:57.178-0700 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2024-03-04T21:34:57.178-0700 [INFO]  Loading CLI configuration from /User/someone/path/.terraform.d/credentials.tfrc.json
2024-03-04T21:34:57.178-0700 [DEBUG] checking for credentials in "/User/someone/path/.terraform.d/plugins"
2024-03-04T21:34:57.178-0700 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2024-03-04T21:34:57.178-0700 [DEBUG] will search for provider plugins in /User/someone/path/.terraform.d/plugins
2024-03-04T21:34:57.179-0700 [DEBUG] ignoring non-existing provider search directory /User/someone/path/.local/share/terraform/plugins
2024-03-04T21:34:57.179-0700 [DEBUG] ignoring non-existing provider search directory /User/someone/path/.local/share/flatpak/exports/share/terraform/plugins
2024-03-04T21:34:57.179-0700 [DEBUG] ignoring non-existing provider search directory /var/lib/flatpak/exports/share/terraform/plugins
2024-03-04T21:34:57.179-0700 [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2024-03-04T21:34:57.179-0700 [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2024-03-04T21:34:57.179-0700 [DEBUG] ignoring non-existing provider search directory /var/lib/snapd/desktop/terraform/plugins
2024-03-04T21:34:57.179-0700 [INFO]  CLI command args: []string{"get", "-update"}
2024-03-04T21:34:57.182-0700 [DEBUG] Module installer: begin cluster
Downloading s3::https://s3.amazonaws.com/redacted-s3-bucket-name/some-module-name.tar.gz for cluster...
2024/03/04 21:34:57 ERROR: failed to create session with AWS_SDK_LOAD_CONFIG enabled. Use session.NewSession to handle errors occurring during session creation. Error: profile "REDACTED" is configured to use SSO but is missing required configuration: sso_region, sso_start_url
2024-03-04T21:34:57.184-0700 [DEBUG] Module installer: begin workers
Downloading s3::https://s3.amazonaws.com/redacted-s3-bucket-name/some-module-name.tar.gz for workers...
2024/03/04 21:34:57 ERROR: failed to create session with AWS_SDK_LOAD_CONFIG enabled. Use session.NewSession to handle errors occurring during session creation. Error: profile "REDACTED" is configured to use SSO but is missing required configuration: sso_region, sso_start_url
╷
│ Error: Failed to download module
│
│   on main.tf line 5:
│    5: module "cluster" {
│
│ Could not download module "cluster" (main.tf:5) source code from "s3::https://s3.amazonaws.com/redacted-s3-bucket-name/some-module-name.tar.gz": profile "REDACTED" is
│ configured to use SSO but is missing required configuration: sso_region, sso_start_url
╵

╷
│ Error: Failed to download module
│
│   on main.tf line 25:
│   25: module "workers" {
│
│ Could not download module "workers" (main.tf:25) source code from "s3::https://s3.amazonaws.com/redacted-s3-bucket-name/some-module-name.tar.gz": profile
│ "REDACTED" is configured to use SSO but is missing required configuration: sso_region, sso_start_url

Expected Behavior

Should support profiles configured with: https://docs.aws.amazon.com/cli/latest/userguide/sso-configure-profile-token.html

Actual Behavior

Only supports profiles configured with: https://docs.aws.amazon.com/cli/latest/userguide/sso-configure-profile-legacy.html

Steps to Reproduce

Put a module in s3
Try to download it when using an AWS IAM Identity Center profile configured with refreshable tokens

Additional Context

I believe the issue is similar to https://github.com/hashicorp/terraform/issues/32465

References

No response

crw commented 3 months ago

As I believe this functionality relies on go-getter, it is likely related to: https://github.com/hashicorp/go-getter/issues/323

endzyme commented 3 months ago

As I believe this functionality relies on go-getter, it is likely related to: hashicorp/go-getter#323

Anything I can help with to get this updated @crw?

crw commented 3 months ago

@endzyme Unfortunately not. go-getter is essentially frozen except in exceptional circumstances. There are a number of open AWS issues (https://github.com/hashicorp/go-getter/pull/467, https://github.com/hashicorp/go-getter/pull/457). I am linking these via this comment on the off chance they can be resolved together, if the SDKv2 issue gets resolved. Thanks!

hashicorp / terraform