search

hashicorp / terraform

Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
https://www.terraform.io/
Other
42.55k stars 9.53k forks source link

Error in creating ETS pipeline. AccessDeniedException #15369

Closed N4R3N closed 7 years ago

N4R3N commented 7 years ago

Provider

AWS

Terraform Version

v0.9.8

tf file

resource "aws_elastictranscoder_pipeline" "hls" {
  input_bucket = "${data.terraform_remote_state.global.video_input_bucket_id}"
  name         = "${var.environment}-ets-hls-pipeline"
  role         = "${data.terraform_remote_state.global.transcode_role_arn}"

  content_config = {
    bucket        = "${data.terraform_remote_state.global.video_output_bucket_id}"
    storage_class = "Standard"
  }

  thumbnail_config = {
    bucket        = "${data.terraform_remote_state.global.thumbnail_bucket_id}"
    storage_class = "Standard"
  }
  notifications {
    progressing = "${aws_sns_topic.ElasticTranscoderProcessingTopic.arn}"
    warning = "${aws_sns_topic.ElasticTranscoderWarningTopic.arn}"
    error = "${aws_sns_topic.ElasticTranscoderErrorTopic.arn}"
    completed = "${aws_sns_topic.ElasticTranscoderCompletedTopic.arn}"
  }

}

Expected Behavior

Create ETS pipeline while using the keys of a user with admin privileges.

Actual Behavior

aws_elastictranscoder_pipeline.hls: Error creating Elastic Transcoder Pipeline: AccessDeniedException: User: arn:aws:iam::12345678901:user/abcd is not authorized to access this resource status code: 403, request id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Steps to Reproduce

  1. terraform apply

Important Factoids

Using the same user keys I can create the pipeline through AWS CLI, but Terraform v0.9.8 fails creating the pipeline throwing a 403 Using S3 as the remote backend with a state file for VPC and its resources and a remote state file for the buckets and iam resources All other resources viz. SNS, IAM role got created in the same terraform apply but creating the transcode pipeline failed

Debug logs

2017/06/22 16:17:40 [DEBUG] plugin: terraform: aws-provider (internal) 2017/06/22 16:17:40 [DEBUG] [aws-sdk-go] DEBUG: Request sns/GetTopicAttributes Details:
2017/06/22 16:17:40 [DEBUG] plugin: terraform: ---[ REQUEST POST-SIGN ]-----------------------------
2017/06/22 16:17:40 [DEBUG] plugin: terraform: POST / HTTP/1.1
2017/06/22 16:17:40 [DEBUG] plugin: terraform: Host: sns.eu-west-1.amazonaws.com
2017/06/22 16:17:40 [DEBUG] plugin: terraform: User-Agent: aws-sdk-go/1.8.34 (go1.8; linux; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.9.8
2017/06/22 16:17:40 [DEBUG] plugin: terraform: Content-Length: 122
2017/06/22 16:17:40 [DEBUG] plugin: terraform: Authorization: AWS4-HMAC-SHA256 Credential=AAAAAAAAA11AAAAAAAAA/20170622/eu-west-1/sns/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date, Signature=0000000000000000000000000000000000000000000000000000000000000000
2017/06/22 16:17:40 [DEBUG] plugin: terraform: Content-Type: application/x-www-form-urlencoded; charset=utf-8
2017/06/22 16:17:40 [DEBUG] plugin: terraform: X-Amz-Date: 20170622T161740Z
2017/06/22 16:17:40 [DEBUG] plugin: terraform: Accept-Encoding: gzip
2017/06/22 16:17:40 [DEBUG] plugin: terraform:
2017/06/22 16:17:40 [DEBUG] plugin: terraform: Action=GetTopicAttributes&TopicArn=arn%3Aaws%3Asns%3Aeu-west-1%3A111111111111%3Adev-ets-completed-topic&Version=2010-03-31

2017/06/22 16:17:47 [DEBUG] plugin: terraform: aws-provider (internal) 2017/06/22 16:17:47 [DEBUG] [aws-sdk-go] DEBUG: Request ec2/DescribeAccountAttributes Details:
2017/06/22 16:17:47 [DEBUG] plugin: terraform: ---[ REQUEST POST-SIGN ]-----------------------------
2017/06/22 16:17:47 [DEBUG] plugin: terraform: POST / HTTP/1.1
2017/06/22 16:17:47 [DEBUG] plugin: terraform: Host: ec2.eu-west-1.amazonaws.com
2017/06/22 16:17:47 [DEBUG] plugin: terraform: User-Agent: aws-sdk-go/1.8.34 (go1.8; linux; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.9.8
2017/06/22 16:17:47 [DEBUG] plugin: terraform: Content-Length: 87
2017/06/22 16:17:47 [DEBUG] plugin: terraform: Authorization: AWS4-HMAC-SHA256 Credential=AAAAAAAAA11AAAAAAAAA/20170622/eu-west-1/ec2/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-date, Signature=1111111111111111112121212121212121212122212121231212122121212121
2017/06/22 16:17:47 [DEBUG] plugin: terraform: Content-Type: application/x-www-form-urlencoded; charset=utf-8
2017/06/22 16:17:47 [DEBUG] plugin: terraform: X-Amz-Date: 20170622T161747Z
2017/06/22 16:17:47 [DEBUG] plugin: terraform: Accept-Encoding: gzip
2017/06/22 16:17:47 [DEBUG] plugin: terraform:
2017/06/22 16:17:47 [DEBUG] plugin: terraform: Action=DescribeAccountAttributes&AttributeName.1=supported-platforms&Version=2016-11-15

2017/06/22 16:17:47 [DEBUG] plugin: terraform: aws-provider (internal) 2017/06/22 16:17:47 [DEBUG] [aws-sdk-go] DEBUG: Request elastictranscoder/CreatePipeline Details:
2017/06/22 16:17:47 [DEBUG] plugin: terraform: ---[ REQUEST POST-SIGN ]-----------------------------
2017/06/22 16:17:47 [DEBUG] plugin: terraform: POST /2012-09-25/pipelines HTTP/1.1
2017/06/22 16:17:47 [DEBUG] plugin: terraform: Host: elastictranscoder.eu-west-1.amazonaws.com
2017/06/22 16:17:47 [DEBUG] plugin: terraform: User-Agent: aws-sdk-go/1.8.34 (go1.8; linux; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.9.8
2017/06/22 16:17:47 [DEBUG] plugin: terraform: Content-Length: 549
2017/06/22 16:17:47 [DEBUG] plugin: terraform: Authorization: AWS4-HMAC-SHA256 Credential=AAAAAAAAA11AAAAAAAAA/20170622/eu-west-1/elastictranscoder/aws4_request, SignedHeaders=content-length;host;x-amz-date, Signature=2222222222222222222222222222222222222222222222222222222222222222
2017/06/22 16:17:47 [DEBUG] plugin: terraform: X-Amz-Date: 20170622T161747Z
2017/06/22 16:17:47 [DEBUG] plugin: terraform: Accept-Encoding: gzip
2017/06/22 16:17:47 [DEBUG] plugin: terraform:
2017/06/22 16:17:47 [DEBUG] plugin: terraform: {"InputBucket":"dev-transcode-in","Name":"dev-ets-hls-pipeline","Notifications":{"Completed":"arn:aws:sns:eu-west-1:111111111111:dev-ets-completed-topic","Error":"arn:aws:sns:eu-west-1:111111111111:dev-ets-error-opic","Progressing":"arn:aws:sns:eu-west-1:111111111111:dev-ets-processing-topic","Warning":"arn:aws:sns:eu-west-1:111111111111:dev-ets-warning-topic"},"OutputBucket":"dev-transcode-out","Role":"arn:aws:s3:::dev-code-versioned","ThumbnailConfig":{"Bucket":"dev-transcode-thumbnail","StorageClass":"Standard"}}
2017/06/22 16:17:47 [DEBUG] plugin: terraform: -----------------------------------------------------
2017/06/22 16:17:48 [DEBUG] plugin: terraform: aws-provider (internal) 2017/06/22 16:17:48 [DEBUG] [aws-sdk-go] DEBUG: Response elastictranscoder/CreatePipeline Details:
2017/06/22 16:17:48 [DEBUG] plugin: terraform: ---[ RESPONSE ]--------------------------------------
2017/06/22 16:17:48 [DEBUG] plugin: terraform: HTTP/1.1 403 Forbidden
2017/06/22 16:17:48 [DEBUG] plugin: terraform: Connection: close
2017/06/22 16:17:48 [DEBUG] plugin: terraform: Content-Length: 104
2017/06/22 16:17:48 [DEBUG] plugin: terraform: Content-Type: application/json
2017/06/22 16:17:48 [DEBUG] plugin: terraform: Date: Thu, 22 Jun 2017 16:17:47 GMT
2017/06/22 16:17:48 [DEBUG] plugin: terraform: X-Amzn-Errortype: AccessDeniedException:
2017/06/22 16:17:48 [DEBUG] plugin: terraform: X-Amzn-Requestid: 54e20ae6-5766-11e7-8c44-c38546fd909e
2017/06/22 16:17:48 [DEBUG] plugin: terraform:
2017/06/22 16:17:48 [DEBUG] plugin: terraform:

Something fishy with the post for elastictranscoder/CreatePipeline and the debug log shows it as 2017/06/22 16:17:47 [DEBUG] plugin: terraform: POST /2012-09-25/pipelines HTTP/1.1

N4R3N commented 7 years ago

Found the actual culprit, it is not an issue with Terraform. The role was mapped wrongly to a bucket policy. It is a very dubious error from AWS. I am still baffled as to why it is 403 and not a 400. Anyway it is not an issue so closing it.

ghost commented 4 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.