aws_iot_topic_rule modification fails with UnauthorizedException

jamescrowley commented 4 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.12.24

provider.archive v1.3.0
provider.aws v2.54.0
provider.null v2.1.2

Affected Resource(s)

aws_iot_topic_rule

Terraform Configuration Files

resource "aws_iot_topic_rule" "forward" {
  name        = "${var.prefix}_forward_${var.name}"
  enabled     = true
  sql         = "SELECT * FROM 'v2/+/data/${var.name}'"
  sql_version = "2015-10-08"

  dynamic kinesis {
    for_each = var.realtime_lambda == "" ? [] : [var.realtime_lambda]
    content {
      partition_key = "$${topic()}"
      role_arn      = var.role_iot.arn
      stream_name   = aws_kinesis_stream.realtime_stream[0].name
    }
  }

  firehose {
    role_arn             = var.role_iot.arn
    delivery_stream_name = aws_kinesis_firehose_delivery_stream.to_s3_parquet.name
  }
}

Debug Output

2020-03-25T05:55:28.650Z [DEBUG] plugin.terraform-provider-aws_v2.54.0_x4: 2020/03/25 05:55:28 [DEBUG] [aws-sdk-go] DEBUG: Validate Response iot/GetTopicRule failed, attempt 0/25, error UnauthorizedException: Access to topic rule 'sandbox_v2_forward_state_data' was denied

Available in terraform cloud: https://app.terraform.io/app/Relectrify/workspaces/Web-Infrastructure-sandbox_v2/runs/run-KTwRTs996B3LxVif

Expected Behavior

Shouldn't encounter UnauthorizedException errors

Actual Behavior

aws_iot_topic_rule modification fails with UnauthorizedException

Steps to Reproduce

This was working fine, but our most recent change to the terraform config appears to have triggered this state, and it consistently fails running terraform plan

Running in terraform cloud: https://app.terraform.io/app/Relectrify/workspaces/Web-Infrastructure-sandbox_v2/runs/run-KTwRTs996B3LxVif

Important Factoids

References

Appears to be closely related to this issue which was fixed in January https://github.com/terraform-providers/terraform-provider-aws/issues/10365

JoshM1994 commented 3 years ago

I encountered a similar issue on v2.47

I was having issues updating an error action - terraform apply would always see the change but the rule never got updated. Eventually, I removed the rule manually and ran terraform rm [resource] to remove it from the state.

Upon running a fresh terraform apply, the IoT rule was successfully created with the changes I needed

justinretzolk commented 3 years ago

Hey @jamescrowley 👋 Thank you for taking the time to file this issue. Given that there's been a number of Terraform and AWS provider releases since you initially filed it, can you confirm if you're still experiencing this behavior?

tomscholz commented 2 years ago

I can confirm that this bug is still occurring 👍

richardj-bsquare commented 2 years ago

Yep, still get this on occasions; only to fix is to remove topic rule state or topic rule.

Error: error getting IoT Topic Rule (iot_ingester): UnauthorizedException: Access to topic rule 'iot_ingester' was denied

justinretzolk commented 2 years ago

Hey y'all 👋 Looking at the error, the UnauthorizedException is coming from AWS. Looking at the AWS Go SDK reference for GetTopicRule (the function that is being called when this error is logged), it calls out:

Requires permission to access the GetTopicRule (https://docs.aws.amazon.com/service-authorization/latest/reference/list_awsiot.html#awsiot-actions-as-permissions) action.

Can you verify that the credentials being used to authenticate with the AWS Provider has the necessary permissions to access GetTopicRule?

tomscholz commented 2 years ago

The permission is definitely there. As @JoshM1994 already mentioned does it work eventually, but not always.

I encountered a similar issue on v2.47

I was having issues updating an error action - terraform apply would always see the change but the rule never got updated. Eventually, I removed the rule manually and ran terraform rm [resource] to remove it from the state.

Upon running a fresh terraform apply, the IoT rule was successfully created with the changes I needed

ewbankkit commented 2 years ago

This error is caused by the fact that GetTopicRule returns UnauthorizedException if the rule no longer exists (rather than a useful error like ResourceNotFoundException). https://github.com/hashicorp/terraform-provider-aws/pull/24395 has fixed this by first listing all topic rules and only attempting to get rule details if the rule still exists.

github-actions[bot] commented 2 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform-provider-aws