[Bug]: aws_lambda_event_source_mapping returns ValidationExcpetion for self managed apache kafka

[Flaudiepie]

Terraform Core Version

1.3.4

AWS Provider Version

4.37.0

Affected Resource(s)

aws_lambda_event_source_mapping

Expected Behavior

terraform apply should push the resource to aws. The event source is configured for a selfmanaged apache kafka.

Actual Behavior

When applying a Validation Exception is thrown with no explicit reason

Relevant Error/Panic Output Snippet

aws_lambda_event_source_mapping.lambda_kafka_confluent_cloud_source: Creating...

Error: error creating Lambda Event Source Mapping (Self-Managed Apache Kafka): ValidationException: 
  status code: 400, request id: 8b5950e8-4f46-4c57-a9f8-6af039f84d66
  with aws_lambda_event_source_mapping.lambda_kafka_confluent_cloud_source,
  on lambda.tf line 36, in resource "aws_lambda_event_source_mapping" 
  "lambda_kafka_confluent_cloud_source": 
  36: resource "aws_lambda_event_source_mapping" "lambda_kafka_confluent_cloud_source" {

Terraform Configuration Files

resource "aws_lambda_event_source_mapping" "lambda_kafka_confluent_cloud_source" {
  function_name     = aws_lambda_function.ekr_backporter.arn
  topics            = var.kafka_topics
  batch_size        = 100
  enabled           = true
  starting_position = "LATEST"

  self_managed_event_source {
    endpoints = {
      KAFKA_BOOTSTRAP_SERVERS = var.kafka_endpoint
    }
  }

  self_managed_kafka_event_source_config {
    consumer_group_id = var.kafka_consumer_group_id
  }

  source_access_configuration {
    type = "CLIENT_CERTIFICATE_TLS_AUTH"
    uri  = aws_secretsmanager_secret.kafka_credentials.arn
  }

  source_access_configuration {
    type = "VPC_SECURITY_GROUP"
    uri  = "security_group:${aws_default_security_group.default_security_group.id}"
  }

  source_access_configuration {
    type = "VPC_SUBNET"
    uri  = "subnet:${aws_subnet.subnet_private.id}"
  }
}

Steps to Reproduce

1. add resource to code (see terraform configuration file)
2. perform terraform apply

Debug Output

debug.txt

Panic Output

No response

Important Factoids

The resource is configured for a self managed kafka. We tried playing around with the single configuration values but nothing worked.

We also tried to setup the event source via the aws console with the same values and it worked. For validation we compared the aws console and terraform requests and they were completely identical besides some formatting.

Importing from the aws console and correcting the local resource also does not work.

References

https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_event_source_mapping

Would you like to implement a fix?

No response

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[github-actions[bot]]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!