Open sklirg opened 1 year ago
Voting for Prioritization
Volunteering to Work on This Issue
I have also experienced this same behavior in provider version 5.9.0 with SQS and with Amazon Keyspace resources.
Policy attempts tag-based resource restrictions as above, but terraform calls the Get call faster than the tag relationship is established and errors out after the first call. Permissions have been tested in AWS console and CLI with no permission issues. Trace dump from keyspaces example.
[DEBUG] provider.terraform-provider-aws_v5.9.0_x5: HTTP Request Sent: aws.operation=CreateKeyspace aws.sdk=aws-sdk-go-v2 http.request.header.amz_sdk_request="attempt=1; max=25" http.url=https://cassandra.us-east-1.amazonaws.com/ tf_provider_addr=registry.terraform.io/hashicorp/aws http.request.header.x_amz_target=KeyspacesService.CreateKeyspace @module=aws aws.service=Keyspaces http.method=POST http.request.body="{"keyspaceName":"my_keyspace","tags":[{"key":"Name","value":"Provider Tag"},{"key":"Domain","value":"001"},{"key":"Environment","value":"Testprod"}]}
" http.request.header.amz_sdk_invocation_id=0e119d5c-5748-4b71-903d-bf70ca85bd28 http.request.header.authorization="AWS4-HMAC-SHA256 Credential=*****/us-east-1/ca********aws4_request, Sign*********=amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-date;x-amz-target, Signature=*****" http.request_content_length=149 tf_req_id=42b1a992-2105-e3de-0715-6aa684ffb3f5 http.request.header.content_type=application/x-amz-json-1.0 http.user_agent="APN/1.0 HashiCorp/1.0 Terraform/1.4.6 (+https://www.terraform.io/) terraform-provider-aws/5.9.0 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go-v2/1.19.0 os/macos lang/go#1.20.5 md/GOOS#darwin md/GOARCH#amd64 api/keyspaces#1.3.3" @caller=github.com/hashicorp/aws-sdk-go-base/v2@v2.0.0-beta.32/logging/logger.go:39 aws.region=us-east-1 http.request.header.x_amz_date=20230814T172922Z net.peer.name=cassandra.us-east-1.amazonaws.com tf_mux_provider=*schema.GRPCProviderServer tf_resource_type=aws_keyspaces_keyspace tf_rpc=ApplyResourceChange timestamp=2023-08-14T12:29:22.764-0500
[DEBUG] provider.terraform-provider-aws_v5.9.0_x5: HTTP Response Received: http.response.header.content_type=application/x-amz-json-1.0 http.response.header.date="Mon, 14 Aug 2023 17:29:23 GMT" http.status_code=200 @module=aws aws.operation=CreateKeyspace aws.sdk=aws-sdk-go-v2 aws.service=Keyspaces http.duration=261 tf_provider_addr=registry.terraform.io/hashicorp/aws tf_rpc=ApplyResourceChange aws.region=us-east-1 http.response_content_length=81 tf_mux_provider=*schema.GRPCProviderServer tf_req_id=42b1a992-2105-e3de-0715-6aa684ffb3f5 @caller=github.com/hashicorp/aws-sdk-go-base/v2@v2.0.0-beta.32/logging/logger.go:39 http.response.body="{"resourceArn":"arn:aws:cassandra:us-east-1:<ACCOUNTID>:/keyspace/my_keyspace/"}
" http.response.header.x_amzn_requestid=0af2af4a-dc35-4eae-b459-6665c9ca1314 tf_resource_type=aws_keyspaces_keyspace timestamp=2023-08-14T12:29:23.026-0500
[DEBUG] provider.terraform-provider-aws_v5.9.0_x5: [DEBUG] Waiting for state to become: [success]
[DEBUG] provider.terraform-provider-aws_v5.9.0_x5: HTTP Request Sent: http.request.header.amz_sdk_invocation_id=6154f7cd-5b75-454c-af93-b53375c2cece http.url=https://cassandra.us-east-1.amazonaws.com/ http.user_agent="APN/1.0 HashiCorp/1.0 Terraform/1.4.6 (+https://www.terraform.io/) terraform-provider-aws/5.9.0 (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go-v2/1.19.0 os/macos lang/go#1.20.5 md/GOOS#darwin md/GOARCH#amd64 api/keyspaces#1.3.3" net.peer.name=cassandra.us-east-1.amazonaws.com @caller=github.com/hashicorp/aws-sdk-go-base/v2@v2.0.0-beta.32/logging/logger.go:39 @module=aws tf_req_id=42b1a992-2105-e3de-0715-6aa684ffb3f5 http.method=POST http.request.body="{"keyspaceName":"my_keyspace"}
" http.request.header.content_type=application/x-amz-json-1.0 aws.operation=GetKeyspace http.request.header.x_amz_target=KeyspacesService.GetKeyspace http.request_content_length=30 tf_mux_provider=*schema.GRPCProviderServer http.request.header.authorization="AWS4-HMAC-SHA256 Credential=*****/us-east-1/ca********aws4_request, Sign*********=amz-sdk-invocation-id;amz-sdk-request;content-length;content-type;host;x-amz-date;x-amz-target, Signature=*****" http.request.header.x_amz_date=20230814T172923Z tf_provider_addr=registry.terraform.io/hashicorp/aws tf_resource_type=aws_keyspaces_keyspace aws.region=us-east-1 aws.sdk=aws-sdk-go-v2 aws.service=Keyspaces http.request.header.amz_sdk_request="attempt=1; max=25" tf_rpc=ApplyResourceChange timestamp=2023-08-14T12:29:23.026-0500
[DEBUG] provider.terraform-provider-aws_v5.9.0_x5: HTTP Response Received: aws.region=us-east-1 aws.service=Keyspaces http.response.header.content_type=application/x-amz-json-1.0 http.status_code=400 @caller=github.com/hashicorp/aws-sdk-go-base/v2@v2.0.0-beta.32/logging/logger.go:39 http.duration=52 http.response.header.x_amzn_requestid=ea819870-9fd7-4a1a-bd9e-dead38d9025d @module=aws http.response_content_length=191 tf_resource_type=aws_keyspaces_keyspace tf_rpc=ApplyResourceChange aws.sdk=aws-sdk-go-v2 http.response.body="{"__type":"com.amazon.hele*********dend#Acce*************tion","message":"User arn:aws:iam::<ACCOUNTID>:user/tf_user has no Select permission on keyspace my_keyspace or any of its parents."}
Terraform Core Version
1.5.4
AWS Provider Version
5.4.0
Affected Resource(s)
Expected Behavior
Expecting Terraform to not mark the resource as tainted if it on a 2nd plan shows "no changes" (after manually removing the taint)
Actual Behavior
Applying the resource results in an error, where Terraform receives AccessDenied:
Relevant Error/Panic Output Snippet
No response
Terraform Configuration Files
IAM Policy for the role in use:
Terraform:
Steps to Reproduce
Notice that the resource is tainted. (Verify in AWS that the queue looks as expected)
Untaint the resource
Terraform plan shows no changes
Debug Output
Partial output from apply step:
Panic Output
No response
Important Factoids
A lot of Terraform examples use IAM Policies with
resources = ["*"]
orresources = [arn]
, but here we are using conditions based on resource tags.One workaround I can think of is to allow
sqs:GetQueueAttributes
onresources = ["*"]
, but that will bypass the access restrictions we're trying to impose using tags.The motivation for doing it this way is to provide a Terraform Module where a user can create, own and manage SQS Queues with a scoped-down IAM Role, but not interfere with other SQS queues than their own. (We are using this in a CI/CD pipeline for an application, where the application has a "deployment profile" in IAM which allows it to access resources based on tags, namely
Service="service-name"
.)References
I think this https://github.com/hashicorp/terraform-provider-aws/issues/13980 and this https://github.com/hashicorp/terraform-provider-aws/issues/27393 might be related, in case the tags are not propagated to the resource before the API request is sent, which results in an AccessDenied because the resource doesn't match the IAM statement condition.
Would you like to implement a fix?
None