kms:DescribeKey on default S3 KMS key should not be required

FireballDWF commented 6 years ago

Hi there,

Thank you for opening an issue. Please note that we try to keep the Terraform issue tracker reserved for bug reports and feature requests. For general usage questions, please see: https://www.terraform.io/community.html.

Terraform Version

Terraform 0.10.8 plugin.terraform-provider-aws_v1.9.0_x4

Affected Resource(s)

Please list the resources as a list, for example:

aws_s3_bucket_object

If this issue appears to affect multiple resources, it may be an issue with Terraform's core, so please mention this.

Terraform Configuration Files

provider aws {
  version = "~> 1.9"
}
resource "aws_s3_bucket_object" "emr_server_cert_object" {
  key                    = "${var.tag_application_name}-${var.name_suffix}-emr_server-cert.zip"
  bucket                 = "${var.app_bucket}"
  source                 = "${path.module}/emr_server_cert.zip"
  server_side_encryption = "aws:kms"
  kms_key_id             = "${var.kms_key_arn}"
}

Debug Output

https://gist.github.com/55e126e62d3c205c1082891c700fdac5

Expected Behavior

terraform-provider-aws should have only called kms:DescribeKey on the value of the var.kms_key_arn which was "arn:aws:kms:us-east-1:253038043104:key/b0c45cd7-29fa-4b01-a24b-f2bbacbc385f"

Actual Behavior

Instead it called it on arn:aws:kms:us-east-1:253038043104:key/9e96e7f3-34f5-4db0-a0ba-b22dea13e4d7 which is the default S3 KMS Key. No where in the bucket is the default S3 key used, and the role being used does not have or need access to the default S3 KMS key. The error received is:

default S3 KMS key (alias/aws/s3): AccessDeniedException: User: arn:aws:sts::253038043104:assumed-role/adfs-iamtestingdeveloper/vault-ldap-trpf439-aws-trp-ciam-np-253038043104-1519075546-9788 is not authorized to perform: kms:DescribeKey on resource: arn:aws:kms:us-east-1:253038043104:key/9e96e7f3-34f5-4db0-a0ba-b22dea13e4d7

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

Setup IAM role which has an explict deny on kms:DescribeKey for the default KMS S3 key.
Setup an S3 bucket with a default encryption policy aws:kms with the same kms_key_arn as will be used in the aws_s3_bucket_object.
Use aws_s3_bucket_object to put a file in that same bucket and specify the kms_key_id (same as when setting up the default bucket encryption).
Observe the error

FireballDWF commented 6 years ago

I'm working on getting a gist of the trace.log, but in the meantime here is a grep showing some relevant lines:

$ grep 'arn:aws:kms:us-east-1:253038043104:key/' trace.log 
2018-02-19T16:33:49.420-0500 [DEBUG] plugin.terraform-provider-aws_v1.9.0_x4: <ServerSideEncryptionConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Rule><ApplyServerSideEncryptionByDefault><SSEAlgorithm>aws:kms</SSEAlgorithm><KMSMasterKeyID>arn:aws:kms:us-east-1:253038043104:key/b0c45cd7-29fa-4b01-a24b-f2bbacbc385f</KMSMasterKeyID></ApplyServerSideEncryptionByDefault></Rule></ServerSideEncryptionConfiguration>
2018-02-19T16:33:49.420-0500 [DEBUG] plugin.terraform-provider-aws_v1.9.0_x4:           KMSMasterKeyID: "arn:aws:kms:us-east-1:253038043104:key/b0c45cd7-29fa-4b01-a24b-f2bbacbc385f",
2018-02-19T16:33:49.736-0500 [DEBUG] plugin.terraform-provider-aws_v1.9.0_x4: X-Amz-Server-Side-Encryption-Aws-Kms-Key-Id: arn:aws:kms:us-east-1:253038043104:key/b0c45cd7-29fa-4b01-a24b-f2bbacbc385f
2018-02-19T16:33:49.737-0500 [DEBUG] plugin.terraform-provider-aws_v1.9.0_x4:   SSEKMSKeyId: "arn:aws:kms:us-east-1:253038043104:key/b0c45cd7-29fa-4b01-a24b-f2bbacbc385f",
2018-02-19T16:33:49.838-0500 [DEBUG] plugin.terraform-provider-aws_v1.9.0_x4: 2018/02/19 16:33:49 [DEBUG] [aws-sdk-go] {"__type":"AccessDeniedException","Message":"User: arn:aws:sts::253038043104:assumed-role/adfs-iamtestingdeveloper/vault-ldap-trpf439-aws-trp-ciam-np-253038043104-1519075546-9788 is not authorized to perform: kms:DescribeKey on resource: arn:aws:kms:us-east-1:253038043104:key/9e96e7f3-34f5-4db0-a0ba-b22dea13e4d7"}
2018-02-19T16:33:49.838-0500 [DEBUG] plugin.terraform-provider-aws_v1.9.0_x4: 2018/02/19 16:33:49 [DEBUG] [aws-sdk-go] DEBUG: Validate Response kms/DescribeKey failed, not retrying, error AccessDeniedException: User: arn:aws:sts::253038043104:assumed-role/adfs-iamtestingdeveloper/vault-ldap-trpf439-aws-trp-ciam-np-253038043104-1519075546-9788 is not authorized to perform: kms:DescribeKey on resource: arn:aws:kms:us-east-1:253038043104:key/9e96e7f3-34f5-4db0-a0ba-b22dea13e4d7
2018/02/19 16:33:49 [ERROR] root.emr-cluster-test: eval: *terraform.EvalRefresh, err: aws_s3_bucket_object.emr_server_cert_object: Failed to describe default S3 KMS key (alias/aws/s3): AccessDeniedException: User: arn:aws:sts::253038043104:assumed-role/adfs-iamtestingdeveloper/vault-ldap-trpf439-aws-trp-ciam-np-253038043104-1519075546-9788 is not authorized to perform: kms:DescribeKey on resource: arn:aws:kms:us-east-1:253038043104:key/9e96e7f3-34f5-4db0-a0ba-b22dea13e4d7
2018/02/19 16:33:49 [ERROR] root.emr-cluster-test: eval: *terraform.EvalSequence, err: aws_s3_bucket_object.emr_server_cert_object: Failed to describe default S3 KMS key (alias/aws/s3): AccessDeniedException: User: arn:aws:sts::253038043104:assumed-role/adfs-iamtestingdeveloper/vault-ldap-trpf439-aws-trp-ciam-np-253038043104-1519075546-9788 is not authorized to perform: kms:DescribeKey on resource: arn:aws:kms:us-east-1:253038043104:key/9e96e7f3-34f5-4db0-a0ba-b22dea13e4d7

FireballDWF commented 6 years ago

Editted Debug Output section to link to gist of trace file

ashishapy commented 6 years ago

I am also getting same error in exact scenario.

Any workaround or solution to this bug?

netdisciple commented 6 years ago

I also have this issue.

brandonstevens commented 6 years ago

I'm able to reproduce this on the latest release (0.24.0). Looks like it's due to how TF tries to keep state consistent when the default S3 key is used. AWS automatically sets the key ID and subsequent reads will return it. Without this code, Terraform plans would think the key changed from "" to the default key ARN. Here is the offending code block (https://github.com/terraform-providers/terraform-provider-aws/blob/master/aws/resource_aws_s3_bucket_object.go#L297):

if resp.SSEKMSKeyId != nil {
    // retrieve S3 KMS Default Master Key
    kmsconn := meta.(*AWSClient).kmsconn
    kmsresp, err := kmsconn.DescribeKey(&kms.DescribeKeyInput{
        KeyId: aws.String("alias/aws/s3"),
    })
    if err != nil {
        return fmt.Errorf("Failed to describe default S3 KMS key (alias/aws/s3): %s", err)
    }

    if *resp.SSEKMSKeyId != *kmsresp.KeyMetadata.Arn {
        log.Printf("[DEBUG] S3 object is encrypted using a non-default KMS Key ID: %s", *resp.SSEKMSKeyId)
        d.Set("kms_key_id", resp.SSEKMSKeyId)
    }
}

I'm wondering if we can simply remove this condition and make kms_key_id a computed value.

Ilhicas commented 5 years ago

On version 0.12.6 still getting the same error as above. A role that only has permission to a given key is being assumed by terraform to Put an encrypted object at S3, resulting in DescribeKey being triggered against the arn for default key.

asahin-icg commented 4 years ago

strange, seems still this isssue isn't solved.

ghost commented 4 years ago

Hitting this issue right now, too: terraform v0.12.28, AWS provider v2.69.0. This prevents us from managing certain configuration blobs using Terraform. Please fix this; the arguments in favour of this behaviour are not good enough to convince our core infra team to grant more permissions on KMS keys.

NavithaK commented 3 years ago

any update on this issue?

laurentmas commented 3 years ago

Same issue for me. We are mid 2021 but still no one fix this...

srinivas-vangari commented 3 years ago

It is Sep-2021, the issue is still there! Can we have a fix for this asap please.

¦ Error: bucket object KMS: Failed to describe default S3 KMS key (alias/aws/s3): AccessDeniedException: User: arn:aws:sts::xxxxxx:assumed-role/xxxxxxxxx is not authorized to perform: kms:DescribeKey on resource: arn:aws:kms:us-east-1:xxxxxxxx:key/<id of the aws/s3 kms key>

Looking at the state file, the kms fields are correctly set.

        "kms_key_id": "arn:aws:kms:us-east-1:xxxxxxx:key/<id of bucket default KMS key, which is not that of aws/s3>",
        "server_side_encryption": "aws:kms",

jasonhuling commented 2 years ago

I just ran into this as well with terraform v1.0.8 and AWS provider 3.62.0. We have default encryption configured using our own KMS Key, not alias/aws/s3, and the object was put successfully and the correct KMS key applied for encryption, but now subsequent plans are failing.