search

hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.77k stars 9.12k forks source link

[Bug]: s3_configuration is not accepted under http_endpoint_configuration #36796

Closed saurabh1611 closed 1 month ago

saurabh1611 commented 6 months ago

Terraform Core Version

1.7.5

AWS Provider Version

5.44.0

Affected Resource(s)

aws_kinesis_firehose_delivery_stream

Expected Behavior

When creating the stream for Dynatrace, I am using http_endpoint_configuration. s3_configuration is a mandatory field but when we pass that within the block terraform gives the following error -

Error: Unsupported block type
│
│   on kinesis_firehose.tf line 15, in resource "aws_kinesis_firehose_delivery_stream" "demo_delivery_stream":
│   15: s3_configuration {
│
│ Blocks of type "s3_configuration" are not expected here.`

Actual Behavior

It should provision the stream with the bucket.

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

resource "aws_kinesis_firehose_delivery_stream" "demo_delivery_stream" {
  name        = "${var.kinesis_stream_name}-delivery"
  destination = "http_endpoint"

  http_endpoint_configuration {
    url                = "https://example.com/my-endpoint"
    name               = "Dynatrace"
    access_key         = "TESTKEYHERE'"
    buffering_size     = 3
    buffering_interval = 60
    role_arn           = aws_iam_role.firehose_role.arn
    s3_backup_mode     = "FailedDataOnly"
    retry_duration = 900

    s3_configuration {
      role_arn           = aws_iam_role.firehose_role.arn
      bucket_arn         = aws_s3_bucket.demo_bucket.arn
      buffering_size     = 10
      buffering_interval = 400
      compression_format = "GZIP"
    }

    request_configuration {
      content_encoding = "GZIP"
    }
  }
}

Steps to Reproduce

terraform plan shows the error.

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

None

github-actions[bot] commented 6 months ago

Community Note

Voting for Prioritization

Volunteering to Work on This Issue

justinretzolk commented 5 months ago

Hey @saurabh1611 👋 Thank you for taking the time to raise this! Your configuration looks valid at first glance, so I think we'll need a bit more information here. Are you able to supply debug logs (redacted as necessary)?

parthyp commented 3 months ago

it looks like there is a bug in docs.

you need to move s3_configuration out of http_endpoint_configuration

we ran into same issue with splunk configuration.

smathangi commented 3 months ago

same error in our plan too │ Error: Unsupported block type │ on resource "aws_kinesis_firehose_delivery_stream" "kinesis_firehose_stream": │ 28: s3_configuration { │ Blocks of type "s3_configuration" are not expected here.

smathangi commented 3 months ago

it looks like it's aws provider version incomparability. For versions < 5.0.0 "s3_configuration" should be outside of http_endpoint_configuration as per the docs. https://registry.terraform.io/providers/hashicorp/aws/4.67.0/docs/resources/kinesis_firehose_delivery_stream#http-endpoint-eg-new-relic-destination

justinretzolk commented 3 months ago

@saurabh1611 can you confirm whether your issue may also have been related to the version, as mentioned above?

dipanjanforexperian commented 3 months ago

I am also facing similar issue while setting up firehose for Splunk. After coming across this thread, I tried keeping the s3_configuration outside. But no luck. code structure I used:

`resource "aws_kinesis_firehose_delivery_stream" "firehose_test_stream" { name = "firehose-test-stream" destination = "splunk"

s3_configuration { role_arn = aws_iam_role.abc.arn bucket_arn = aws_s3_bucket.demo_bucket.arn buffering_size = 10 buffering_interval = 400 compression_format = "GZIP" } splunk_configuration { hec_endpoint = "https://example.com/my-endpoint" hec_token = "####" hec_acknowledgment_timeout = 600 hec_endpoint_type = "Event" s3_backup_mode = "FailedEventsOnly" } }`

Error I am getting:

`Error: Unsupported argument

on kinesis_firehose.tf line 48, in resource "aws_kinesis_firehose_delivery_stream" "firehose_test_stream": 48: buffering_size = 10

An argument named "buffering_size" is not expected here.

Error: Unsupported argument

on kinesis_firehose.tf line 49, in resource "aws_kinesis_firehose_delivery_stream" "firehose_test_stream": 49: buffering_interval = 400

An argument named "buffering_interval" is not expected here.`

justinretzolk commented 1 month ago

Hi all,

Based on the replies here, this comes down to the version of the AWS Provider you're using. This was noted in the version 5.0.0 changelog, and was introduced with https://github.com/hashicorp/terraform-provider-aws/pull/31138.

Since this is expected behavior, I'm going to close this issue. If you experience any additional unexpected behavior, please do raise a new issue and let us know!

github-actions[bot] commented 1 month ago

[!WARNING] This issue has been closed, meaning that any additional comments are hard for our team to see. Please assume that the maintainers will not see them.

Ongoing conversations amongst community members are welcome, however, the issue will be locked after 30 days. Moving conversations to another venue, such as the AWS Provider forum, is recommended. If you have additional concerns, please open a new issue, referencing this one where needed.

github-actions[bot] commented 1 week ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.