A-Shevchenko commented 10 months ago

Terraform Core Version

1.3.9

AWS Provider Version

5.26.0

Affected Resource(s)

data "aws_caller_identity"

Expected Behavior

Succesful resolution of data provider

Actual Behavior

Fails with Credential should be scoped to a valid region.

Relevant Error/Panic Output Snippet

│ Error: reading STS Caller Identity
│
│   with module.production-me-central-1[0].data.aws_caller_identity.current,
│   on modules\regional\main.tf line 28, in data "aws_caller_identity" "current":
│   28: data "aws_caller_identity" "current" {}
│
│ operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: f16ffea6-1579-440e-aed9-f5f18c361a85, api error SignatureDoesNotMatch: Credential should be scoped to a valid region.

Terraform Configuration Files

provider "aws" {
  region = "me-central-1"
  alias = "aws-me-central-1"

  sts_region = "us-east-1"
  endpoints {
    sts = "https://sts.us-east-1.amazonaws.com"
  }
}

data "aws_caller_identity" "current" {}

Steps to Reproduce

Use a provider config with cross-region STS setup as above
Create simple aws_caller_identity data provider

Debug Output

2023-12-11T20:11:36.830+0100 [DEBUG] provider.terraform-provider-aws_v5.26.0_x5.exe: HTTP Request Sent: http.method=POST http.request.header.content_type=application/x-www-form-urlencoded rpc.service=STS @module=aws aws.region=me-central-1 tf_rpc=ReadDataSource http.request.header.x_amz_security_token=***** tf_data_source_type=aws_caller_identity tf_mux_provider=*proto5server.Server rpc.method=GetCallerIdentity http.request.header.amz_sdk_invocation_id=7e636798-3e70-4419-9a59-6d5772f92d61 http.url=https://sts.us-east-1.amazonaws.com/ http.request.header.amz_sdk_request="attempt=1; max=25" http.request_content_length=43 tf_provider_addr=registry.terraform.io/hashicorp/aws rpc.system=aws-api http.request.body="Action=GetCallerIdentity&Version=2011-06-15
" http.request.header.x_amz_date=20231211T191136Z http.request.header.authorization="AWS4-HMAC-SHA256 Credential=ASIA************KIOP/20231211/me-central-1/sts/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-date;x-amz-security-token, Signature=*****" net.peer.name=sts.us-east-1.amazonaws.com tf_req_id=a1b8740e-7b9b-997d-fd33-108a00b55493 @caller=github.com/hashicorp/aws-sdk-go-base/v2@v2.0.0-beta.41/logging/tf_logger.go:45 http.user_agent="APN/1.0 HashiCorp/1.0 Terraform/1.3.9 (+https://www.terraform.io) terraform-provider-aws/5.26.0 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go-v2/1.22.2 os/windows lang/go#1.20.10 md/GOOS#windows md/GOARCH#amd64 api/sts#1.25.1" tf_aws.sdk=aws-sdk-go-v2 timestamp=2023-12-11T20:11:36.830+0100
2023-12-11T20:11:36.939+0100 [DEBUG] provider.terraform-provider-aws_v5.26.0_x5.exe: HTTP Response Received: rpc.method=GetCallerIdentity tf_data_source_type=aws_caller_identity tf_provider_addr=registry.terraform.io/hashicorp/aws tf_rpc=ReadDataSource aws.region=me-central-1 http.response.body="<ErrorResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <Error>
    <Type>Sender</Type>
    <Code>SignatureDoesNotMatch</Code>
    <Message>Credential should be scoped to a valid region. </Message>
  </Error>
  <RequestId>f16ffea6-1579-440e-aed9-f5f18c361a85</RequestId>
</ErrorResponse>
" http.response.header.content_type=text/xml http.response_content_length=300 tf_req_id=a1b8740e-7b9b-997d-fd33-108a00b55493 rpc.service=STS rpc.system=aws-api @module=aws http.duration=108 http.response.header.x_amzn_requestid=f16ffea6-1579-440e-aed9-f5f18c361a85 tf_mux_provider=*proto5server.Server @caller=github.com/hashicorp/aws-sdk-go-base/v2@v2.0.0-beta.41/logging/tf_logger.go:45 http.response.header.date="Mon, 11 Dec 2023 19:11:36 GMT" http.status_code=403 tf_aws.sdk=aws-sdk-go-v2 timestamp=2023-12-11T20:11:36.939+0100

See that Credential of the Authorization header contains me-central-1, while us-east-1 is expected

Panic Output

No response

Important Factoids

As mentioned, it looks like the data provider uses the wrong region in Authorization header. However, in the log few lines above, when configuring the provider itself, the correct sts region is used:

2023-12-11T20:11:36.366+0100 [DEBUG] provider.terraform-provider-aws_v5.26.0_x5.exe: HTTP Request Sent: rpc.system=aws-api http.user_agent="APN/1.0 HashiCorp/1.0 Terraform/1.3.9 (+https://www.terraform.io) terraform-provider-aws/5.26.0 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go-v2/1.22.2 os/windows lang/go#1.20.10 md/GOOS#windows md/GOARCH#amd64 api/sts#1.25.1" http.method=POST http.request.header.x_amz_security_token=***** rpc.service=STS @caller=github.com/hashicorp/aws-sdk-go-base/v2@v2.0.0-beta.41/logging/tf_logger.go:47 aws.region=us-east-1 @module=aws.aws-base http.request.header.content_type=application/x-www-form-urlencoded http.request_content_length=43 http.url=https://sts.us-east-1.amazonaws.com/ tf_provider_addr=registry.terraform.io/hashicorp/aws http.request.body="Action=GetCallerIdentity&Version=2011-06-15
" tf_aws.custom_endpoint=true tf_aws.sdk=aws-sdk-go-v2 tf_rpc=ConfigureProvider http.request.header.x_amz_date=20231211T191136Z tf_mux_provider=*schema.GRPCProviderServer http.request.header.authorization="AWS4-HMAC-SHA256 Credential=ASIA************KIOP/20231211/us-east-1/sts/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-date;x-amz-security-token, Signature=*****" tf_req_id=56cfbe49-a041-eccd-6562-1a9827c53496 http.request.header.amz_sdk_invocation_id=92d45b5b-b84f-4a98-ac39-f238db6702fc rpc.method=GetCallerIdentity http.request.header.amz_sdk_request="attempt=1; max=25" net.peer.name=sts.us-east-1.amazonaws.com timestamp=2023-12-11T20:11:36.366+0100
2023-12-11T20:11:36.822+0100 [DEBUG] provider.terraform-provider-aws_v5.26.0_x5.exe: HTTP Response Received: tf_provider_addr=registry.terraform.io/hashicorp/aws tf_req_id=56cfbe49-a041-eccd-6562-1a9827c53496 @module=aws.aws-base aws.region=us-east-1 tf_mux_provider=*schema.GRPCProviderServer tf_aws.custom_endpoint=true tf_rpc=ConfigureProvider http.response.body="<GetCallerIdentityResponse xmlns="https://sts.amazonaws.com/doc/2011-06-15/">
  <GetCallerIdentityResult>
    <Arn>arn:aws:sts::XXX:assumed-role/CI-SERVER/aws-go-sdk-1702321895875127400</Arn>
    <UserId>AROA*************TP2Y:aws-go-sdk-1702321895875127400</UserId>
    <Account>499704737732</Account>
  </GetCallerIdentityResult>
  <ResponseMetadata>
    <RequestId>4d76f219-3f68-45fb-b7f9-15b7c0c01755</RequestId>
  </ResponseMetadata>
</GetCallerIdentityResponse>
" http.response.header.date="Mon, 11 Dec 2023 19:11:36 GMT" http.status_code=200 rpc.system=aws-api http.response.header.content_type=text/xml http.response_content_length=506 rpc.method=GetCallerIdentity rpc.service=STS tf_aws.sdk=aws-sdk-go-v2 @caller=github.com/hashicorp/aws-sdk-go-base/v2@v2.0.0-beta.41/logging/tf_logger.go:47 http.duration=455 http.response.header.x_amzn_requestid=4d76f219-3f68-45fb-b7f9-15b7c0c01755 timestamp=2023-12-11T20:11:36.822+0100

As you can see, Credential=ASIA************KIOP/20231211/us-east-1/sts/aws4_request, so correct STS region is used - us-east-1

References

No response

Would you like to implement a fix?

None

github-actions[bot] commented 10 months ago

Community Note

Voting for Prioritization

Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
Please see our prioritization guide for information on how we prioritize.
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

If you are interested in working on this issue, please leave a comment.
If this would be your first contribution, please review the contribution guide.

A-Shevchenko commented 10 months ago

BTW, it's a regression issue - doesn't occur on 5.24.0, but occurs starting from 5.25.0, probably related to #34300 as we indeed use AWS config files, but they're OK... UPD: not related to config files, embedding credentials and role ARN into provider config doesn't solve the issue

github-actions[bot] commented 7 months ago

This functionality has been released in v5.38.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

github-actions[bot] commented 6 months ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-provider-aws

[Bug]: Region in Authorization (signature) header doesn't follow sts_region #34865