[Bug]: operation error Pipes: UpdatePipe, https response error StatusCode: 400 validationException: [ECMA 262 regex "^(^arn:aws([a-z]|\-)*:secretsmanager:([a-z]{2}((-gov)|(-iso(b?)))?-[a-z]+-\d{1}):(\d{12}):secret:.+)$" does not match input string "", string "" is too short (length: 0, required minimum: 1)]

[lebohnes]

Terraform Core Version

1.9.3

AWS Provider Version

5.62.0

Affected Resource(s)

resource "aws_pipes_pipe" "msk_consumer" {
  name     = local.resource_name
  role_arn = aws_iam_role.eventbridge_pipe_role.arn
  source   = "smk://${data.aws_msk_cluster.msk_cluster.bootstrap_brokers_sasl_scram}"
  target   = aws_sqs_queue.msk_consumer.arn
  source_parameters {
    self_managed_kafka_parameters {
      vpc {
        security_groups = data.aws_security_groups.msk_cluster.ids
        subnets         = [for s in data.aws_subnet.primary : s.id]
      }
      credentials {
        sasl_scram_512_auth = data.aws_secretsmanager_secret.msk_cluster_credentials.arn
      }
      starting_position = var.starting_position
      topic_name        = var.topic_name
      batch_size        = 10
    }
  filter_criteria {
        dynamic "filter" {
          for_each = local.filters
          content {
            pattern = replace(replace(jsonencode(filter.value), "\\u003c", "<"), "\\u003e", ">")
          }
        }
    }
  log_configuration {
    level = "ERROR"
    cloudwatch_logs_log_destination {
      log_group_arn = aws_cloudwatch_log_group.msk_consumer.arn
    }
  }
  depends_on = [aws_sqs_queue_policy.sqs]
}

Expected Behavior

The resource should update properly, the data.aws_secretsmanager_secret.msk_cluster_credentials.arn is unchanged and should not result in an error

Actual Behavior

Terraform fails to update the resource with an operation error.

Relevant Error/Panic Output Snippet

Error: updating AWS EventBridge Pipes Pipe (resource_name): operation error Pipes: UpdatePipe, https response error StatusCode: 400, RequestID: 1871a660-2b9c-41e6-a032-37c18a663743, ValidationException: [ECMA 262 regex "^(^arn:aws([a-z]|\-)*:secretsmanager:([a-z]{2}((-gov)|(-iso(b?)))?-[a-z]+-\d{1}):(\d{12}):secret:.+)$" does not match input string "", string "" is too short (length: 0, required minimum: 1)]
│
│   with module.msk-consumer-pipe["a4"].module.msk_consumer.aws_pipes_pipe.msk_consumer,
│   on msk\msk_consumer\main.tf line 92, in resource "aws_pipes_pipe" "msk_consumer":
│   92: resource "aws_pipes_pipe" "msk_consumer" {

Terraform Configuration Files

resource "aws_pipes_pipe" "msk_consumer" {
  name     = local.resource_name
  role_arn = aws_iam_role.eventbridge_pipe_role.arn
  source   = "smk://${data.aws_msk_cluster.msk_cluster.bootstrap_brokers_sasl_scram}"
  target   = aws_sqs_queue.msk_consumer.arn
  source_parameters {
    self_managed_kafka_parameters {
      vpc {
        security_groups = data.aws_security_groups.msk_cluster.ids
        subnets         = [for s in data.aws_subnet.primary : s.id]
      }
      credentials {
        sasl_scram_512_auth = data.aws_secretsmanager_secret.msk_cluster_credentials.arn
      }
      starting_position = var.starting_position
      topic_name        = var.topic_name
      batch_size        = 10
    }
  filter_criteria {
        dynamic "filter" {
          for_each = local.filters
          content {
            pattern = replace(replace(jsonencode(filter.value), "\\u003c", "<"), "\\u003e", ">")
          }
        }
    }
  log_configuration {
    level = "ERROR"
    cloudwatch_logs_log_destination {
      log_group_arn = aws_cloudwatch_log_group.msk_consumer.arn
    }
  }
  depends_on = [aws_sqs_queue_policy.sqs]
}

Steps to Reproduce

Create the resource => Update the resource e.g. by updating a filter or an iam role

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[justinretzolk]

Hey @lebohnes 👋 Thank you for taking the time to raise this! When you run a terraform plan, are you seeing any changes to the reported value for sasl_scram_512_auth? If you're able to supply debug logs (redacted as needed), it may be helpful to whoever picks this up.

[lebohnes]

@justinretzolk No, i don't see any changes for sasl_scram_512_auth .. here is the terraform plan output. I hope it helps:

 # module.msk-consumer-pipe["resource_name"].module.msk_consumer.aws_pipes_pipe.msk_consumer will be updated in-place
  ~ resource "aws_pipes_pipe" "msk_consumer" {
        id            = "resource_name"
        name          = "resource_name"
        tags          = {}
        # (9 unchanged attributes hidden)

      ~ source_parameters {
          ~ filter_criteria {
              ~ filter {
                  ~ pattern = jsonencode(
                      ~ {
                          ~ value = {
                              + clusterTime = [
                                  + {
                                      + numeric = [
                                          + ">=",
                                          + 1724768012,
                                        ]
                                    },
                                ]
                              - data        = {
                                  - operationType = [
                                      - "update",
                                    ]
                                }
                            }
                        }
                    )
                }
              - filter {
                  - pattern = jsonencode(
                        {
                          - value = {
                              - clusterTime = [
                                  - {
                                      - numeric = [
                                          - ">=",
                                          - 1724768012,
                                        ]
                                    },
                                ]
                            }
                        }
                    ) -> null
                }
            }

            # (1 unchanged block hidden)
        }

        # (2 unchanged blocks hidden)
    }