Error creating Lambda Function last error: InsufficientRolePermissions: The function's execution role doesn't have permission to perform this operation.

[xposix]

Description

I get this issue when executing the deployment from scratch, but it's fine the second deployment. It seems to be a problem of race condition but I don't know why/where:

│ Error: creating Lambda Function (mea-536-infra-es-create-snapshot): waiting for completion: unexpected state 'Failed', wanted target 'Active'. last error: InsufficientRolePermissions: The function's execution role doesn't have permission to perform this operation.
│ 
│   with module.snapshot_lambda[0].aws_lambda_function.this[0],
│   on .terraform/modules/snapshot_lambda/main.tf line 24, in resource "aws_lambda_function" "this":
│   24: resource "aws_lambda_function" "this" {
│ 
╵

• [x] ✋ I have searched the open/closed issues and my issue is not listed.

⚠️ Note

Before you submit an issue, please perform the following first:

1. Remove the local .terraform directory (! ONLY if state is stored remotely, which hopefully you are following that best practice!): rm -rf .terraform/
2. Re-initialize the project root to pull down modules: terraform init
3. Re-attempt your terraform plan or apply and check if the issue still persists

Versions

• Module version [Required]: 5.3.0
• Terraform version: 1.5.2
• Provider version(s):
• provider registry.terraform.io/hashicorp/archive v2.3.0
• provider registry.terraform.io/hashicorp/aws v4.67.0
• provider registry.terraform.io/hashicorp/external v2.3.1
• provider registry.terraform.io/hashicorp/local v2.4.0
• provider registry.terraform.io/hashicorp/null v3.2.1
• provider registry.terraform.io/hashicorp/random v3.4.3

Reproduction Code [Required]

  count   = var.environment_name == var.shared_environment_name ? 0 : 1
  source  = "terraform-aws-modules/lambda/aws"
  version = "5.3.0"

  publish       = true
  function_name = "${var.environment_name}-infra-es-create-snapshot"
  handler       = "index.lambda_handler"
  runtime       = "python3.10"
  architectures = ["arm64"]

  source_path = "${path.module}/helpers/es_create_snapshot_lambda/"

  vpc_subnet_ids         = data.aws_subnets.private.ids
  vpc_security_group_ids = [aws_security_group.lambda.id]
  attach_network_policy  = true

  allowed_triggers = startswith(var.environment_name, "prod") ? {
    Scheduler = {
      principal  = "events.amazonaws.com"
      source_arn = aws_cloudwatch_event_rule.opensearch_snapshot_to_s3[0].arn
    }
  } : {}

  timeout            = 60
  attach_policy_json = true
  # AWS IAM policy that allows to get SSM parameters and the KMS key they are encrypted with
  # Change this policy to Terraform language

  policy_json = data.aws_iam_policy_document.snapshot_lambda[0].json

  environment_variables = {
    ENVIRONMENT_NAME   = var.environment_name
    SLACK_URL          = var.slack_webhook_url
    SLACK_CHANNEL_NAME = var.slack_channel_name
  }

  tags = local.project_tags
}

data "aws_iam_policy_document" "snapshot_lambda" {
  count = var.environment_name == var.shared_environment_name ? 0 : 1

  statement {
    sid    = "AllowGetSSMParameters"
    effect = "Allow"
    actions = [
      "ssm:GetParameters",
      "ssm:GetParameter"
    ]

    resources = ["*"]
  }

  statement {
    sid    = "AllowGetKMSKey"
    effect = "Allow"
    actions = [
      "kms:Decrypt",
    ]
    resources = local.clz_key_exception == 1 ? [
      "${aws_kms_key.general[0].arn}",
      "${data.aws_kms_key.clz_kms_key.arn}"
    ] : ["${data.aws_kms_key.clz_kms_key.arn}"]
  }

  dynamic "statement" {
    for_each = var.environment_name == var.shared_environment_name ? [] : [1]
    content {
      sid    = "AllowPassRole"
      effect = "Allow"
      actions = [
        "iam:PassRole",
      ]

      resources = [
        aws_iam_role.opensearch_backups[0].arn,
      ]
    }
  }

  statement {
    sid    = "AllowESHttpPut"
    effect = "Allow"
    actions = [
      "es:ESHttpPut",
    ]

    resources = ["${local.elasticsearch_domain_arn}/*"]
  }
}

Steps to reproduce the behavior: No using workspaces. I have cleared the local cache.

Expected behavior

It should deploy in one go.

Actual behavior

The full deployment needs two tries.

Terminal Output Screenshot(s)

╵
╷
│ Error: creating Lambda Function (mea-536-infra-es-create-snapshot): waiting for completion: unexpected state 'Failed', wanted target 'Active'. last error: InsufficientRolePermissions: The function's execution role doesn't have permission to perform this operation.
│ 
│   with module.snapshot_lambda[0].aws_lambda_function.this[0],
│   on .terraform/modules/snapshot_lambda/main.tf line 24, in resource "aws_lambda_function" "this":
│   24: resource "aws_lambda_function" "this" {
│ 
╵
Releasing state lock. This may take a few moments...
make[1]: Leaving directory '/builds/pledge-earth/data-model/infra/1-base'
make[1]: *** [Makefile:107: .apply-core] Error 1

Additional context

The Lambda is deploying on a VPC.

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open 30 days with no activity. Remove stale label or comment or this issue will be closed in 10 days

[github-actions[bot]]

This issue was automatically closed because of stale in 10 days

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.