[Bug]: Cannot update EventBridge Pipe in place when source is DynamoDB Stream

[sweetcoco]

Terraform Core Version

v1.5.2

AWS Provider Version

5.6.2

Affected Resource(s)

aws_pipes_pipe

Expected Behavior

You should be able to update a Pipe in place

Actual Behavior

Any change to a pipe, including tags, that requires an "update in-place" action results in an error.

Relevant Error/Panic Output Snippet

aws_pipes_pipe.dynamodb_stream_pipe: Modifying... [id=my-ddbstream-pipe]
╷
│ Error: updating Amazon EventBridge Pipes Pipe (my-ddbstream-pipe): operation error Pipes: UpdatePipe, https response error StatusCode: 400, RequestID: 8342d00c-730e-434a-8b33-f0e6b23535fb, ValidationException: [numeric instance is lower than the required minimum (minimum: 1, found: 0)]

Terraform Configuration Files

resource "aws_pipes_pipe" "dynamo_stream_pipe" {
  depends_on    = [aws_iam_role_policy.dynamodb_pipe_source, aws_iam_role_policy.sqs_pipe_target]
  name          = "${aws_dynamodb_table.table.name}-pipe"
  role_arn      = aws_iam_role.pipe_role.arn
  source        = aws_dynamodb_table.table.stream_arn
  target        = aws_sqs_queue.ddbstream_queue.arn
  desired_state = "RUNNING"
  tags = {
    "Name"        = "${aws_dynamodb_table.table.name}-pipe"
    "Description" = "this is a description"
  }

  source_parameters {
    dynamodb_stream_parameters {
      batch_size        = 1
      starting_position = "LATEST"
    }
    filter_criteria {
      filter {
        pattern = jsonencode({
          eventName : ["INSERT"]
        })
      }
    }
  }
  target_parameters {}
}

Steps to Reproduce

1. create an aws_dynamodb_table
2. create an aws_pipes_pipe attached the table's stream
3. terraform apply
4. update anything about the pipe, even a tag
5. terraform apply
6. see error

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[justinretzolk]

Hey @sweetcoco 👋 Thank you for taking the time to raise this! So that we have the necessary information in order to look into this, can you supply debug logs (redacted as needed) as well?

[gurudatta-carbon]

getting same error, with kinesis as source, http api destination as target Note: to replicate issue, change anything in target_parameters.

AWS Provider Version: 5.7.0

error: │ Error: updating Amazon EventBridge Pipes Pipe (name): operation error Pipes: UpdatePipe, https response error StatusCode: 400, RequestID: eda9e7a8-014a-44c0-95f3-ef523c070a3c, ValidationException: [numeric instance is lower than the required minimum (minimum: 1, found: 0)]

resource "aws_pipes_pipe" "patient-events-stream-pipe" {
  name     = "name"
  role_arn = var.role_arn
  source = var.source_kinesis_stream_arn
  target = var.api_destination_arn
  source_parameters {
    kinesis_stream_parameters {
      batch_size        = 1
      starting_position = "TRIM_HORIZON"
    }
  }

  target_parameters {
    input_template = <<EOF
       $.data
      EOF
  }
}

[ewbankkit]

Relates https://github.com/hashicorp/terraform-provider-aws/pull/31607.

[gurudatta-carbon]

hello, it seems there is still issue if there is filter parameters addition to top of the given resource i shared.

resource "aws_pipes_pipe" "patient-events-stream-pipe" {
  name     = "name"
  role_arn = var.role_arn
  source = var.source_kinesis_stream_arn
  target = var.api_destination_arn
  source_parameters {
    kinesis_stream_parameters {
      batch_size        = 1
      starting_position = "TRIM_HORIZON"
    }
    filter_criteria {
      filter {
        pattern = jsonencode({
          domain : ["domainName"]
        })
      }
    }
  }

  target_parameters {
    input_template = <<EOF
       $.data
      EOF
  }
}

[sweetcoco]

This issue is not resolved, the same exact error and output in my original opening of the issue is still happening.

Terraform v1.5.2
on darwin_amd64
+ provider registry.terraform.io/hashicorp/aws v5.9.0

[gurudatta-carbon]

can we reopen this issue ? or should i create a new one ?

cc @justinretzolk

[gurudatta-carbon]

also when I tried this combination as well, didn't work, same error, may be just relax the validation rule , if it's in terraform ?

resource "aws_pipes_pipe" "patient-events-stream-pipe" {
  name     = "name"
  role_arn = var.role_arn
  source = var.source_kinesis_stream_arn
  target = var.api_destination_arn
  source_parameters {
    kinesis_stream_parameters {
      batch_size        = 1
      starting_position = "TRIM_HORIZON"
      maximum_record_age_in_seconds = 60
      maximum_retry_attempts        = 3
      dead_letter_config {
        arn = var.sqs_dlq_arn
      }
    }
  }

  target_parameters {
    input_template = <<EOF
       $.data
      EOF
  }
}

[danielMensah]

Can this issue be re-open as I am facing the same error?

Edit: destroying and re-deploying the pipe seems to fix it somehow.

[github-actions[bot]]

This functionality has been released in v5.27.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.