ResourceConflictException: The statement id X provided already exists. Please provide a new statement id ...

alexiskat commented 4 months ago

Description

Whenever a new trigger policy is required (because something has changed in the API GW config) I get the following error: │ Error: adding Lambda Permission (abcd-fd-2d-createGrid-post-api/APIGW-Trigger-fd-2d-createGrid-post): ResourceConflictException: The statement id (APIGW-Trigger-fd-2d-createGrid-post) provided already exists. Please provide a new statement id, or remove the existing statement.

Versions

Module version [Required]: 6.7
Terraform version: terraform 1.7.3
Provider version(s): ➜ terraform providers -version Terraform v1.7.3 on darwin_amd64
provider registry.terraform.io/hashicorp/archive v2.4.2
provider registry.terraform.io/hashicorp/aws v5.38.0
provider registry.terraform.io/hashicorp/external v2.3.3
provider registry.terraform.io/hashicorp/local v2.4.1
provider registry.terraform.io/hashicorp/null v3.2.2
provider registry.terraform.io/hashicorp/random v3.6.0

Reproduction Code [Required]

module "lambda_function" {
  source  = "terraform-aws-modules/lambda/aws"
  version = "6.7"

  function_name = var.lambda_name == null ? "${var.customer_code}-${var.serv}-api" : var.lambda_name
  description   = var.lambda_description == null ? "${var.customer_code}-${var.serv}-api" : var.lambda_description
  handler       = var.lambda_handler
  runtime       = var.lambda_runtime
  publish       = true

  source_path = var.lambda_source_code_path

  store_on_s3 = true
  hash_extra  = "${var.customer_code}-${var.serv}"
  s3_prefix   = var.lambda_build_s3_prefix
  s3_bucket   = var.lambda_code_bucket_id

  layers = var.lambda_layer_arn

  attach_policies    = true
  policies           = [aws_iam_policy.this.arn]
  number_of_policies = 1

  cloudwatch_logs_retention_in_days = var.env == "sandpit" ? 1 : 5
  cloudwatch_logs_tags              = var.tags

  allowed_triggers = {
    ("APIGW-Trigger-${var.serv}") = {
      service    = "apigateway"
      source_arn = format("%s/*/%s%s", var.api_rest_exe_arn, var.api_method, var.api_resource_path)
    }
  }
  create_current_version_allowed_triggers = true
  trigger_on_package_timestamp            = false
  tags                                    = var.tags
}

Steps to reproduce the behavior:

I make a change to the API Gateway resource. For example, I change the HTTP method from a POST to a GET

resource "aws_api_gateway_method" "rest_api_resource_level_2_creategrid_POST" {
  rest_api_id          = aws_api_gateway_rest_api.rest_api.id
  resource_id          = aws_api_gateway_resource.rest_api_resource_level_2_creategrid.id
  http_method          = "POST"
  authorization        = var.api_auth_type
  authorizer_id        = aws_api_gateway_authorizer.api_lambda_auth.id
  authorization_scopes = var.api_auth_method_scope
  api_key_required     = var.api_enable_useage_plan
}

This triggers the creation of a new policy to be created because of var.api_method in the below

  allowed_triggers = {
    ("APIGW-Trigger-${var.serv}") = {
      service    = "apigateway"
      source_arn = format("%s/*/%s%s", var.api_rest_exe_arn, var.api_method, var.api_resource_path)
    }

Expected behavior

Lambda trigger policy to be deleted and a new one created.

Actual behavior

Terraform times outs after 10 minutes of trying to create the new policy and gives an error

Terminal Output Screenshot(s)

module.api_post_fd_2d_createGrid["abcd"].module.lambda_function.aws_lambda_permission.current_version_triggers["APIGW-Trigger-fd-2d-createGrid-post"]: Still creating... [9m40s elapsed]
module.api_post_fd_2d_createGrid["wxyz"].module.lambda_function.aws_lambda_permission.current_version_triggers["APIGW-Trigger-fd-2d-createGrid-post"]: Still creating... [9m40s elapsed]
module.api_post_fd_2d_createGrid["abcd"].module.lambda_function.aws_lambda_permission.current_version_triggers["APIGW-Trigger-fd-2d-createGrid-post"]: Still creating... [9m50s elapsed]
module.api_post_fd_2d_createGrid["wxyz"].module.lambda_function.aws_lambda_permission.current_version_triggers["APIGW-Trigger-fd-2d-createGrid-post"]: Still creating... [9m50s elapsed]
module.api_post_fd_2d_createGrid["wxyz"].module.lambda_function.aws_lambda_permission.current_version_triggers["APIGW-Trigger-fd-2d-createGrid-post"]: Still creating... [10m0s elapsed]
╷
│ Error: adding Lambda Permission (wxyz-fd-2d-createGrid-post-api/APIGW-Trigger-fd-2d-createGrid-post): ResourceConflictException: The statement id (APIGW-Trigger-fd-2d-createGrid-post) provided already exists. Please provide a new statement id, or remove the existing statement.
│ {
│   RespMetadata: {
│     StatusCode: 409,
│     RequestID: "60b83bc6-7d81-4544-b9a4-c37bf45fed32"
│   },
│   Message_: "The statement id (APIGW-Trigger-fd-2d-createGrid-post) provided already exists. Please provide a new statement id, or remove the existing statement.",
│   Type: "User"
│ }
│ 
│   with module.api_post_fd_2d_createGrid["wxyz"].module.lambda_function.aws_lambda_permission.current_version_triggers["APIGW-Trigger-fd-2d-createGrid-post"],
│   on .terraform/modules/api_post_fd_2d_createGrid.lambda_function/main.tf line 243, in resource "aws_lambda_permission" "current_version_triggers":
│  243: resource "aws_lambda_permission" "current_version_triggers" {
│

A workaround is to log into the console and manually delete the policy from Lambda and then Terraform can add the new one. I suspect the issue is with my configuration and not the module but any help with this would be great.

thanks alexis

antonbabenko commented 4 months ago

You should be able to provide and update statement_id (see https://github.com/terraform-aws-modules/terraform-aws-lambda/blob/master/main.tf#L276):

    allowed_triggers = {
    ("APIGW-Trigger-${var.serv}") = {
      service    = "apigateway"
      source_arn = format("%s/*/%s%s", var.api_rest_exe_arn, var.api_method, var.api_resource_path)
      statement_id = format("APIGW-Trigger-%s-%s-%s-%s", var.serv, var.api_rest_exe_arn, var.api_method, var.api_resource_path)  # <- something like this
    }

github-actions[bot] commented 2 months ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

terraform-aws-modules / terraform-aws-lambda