hashicorp / terraform-provider-aws

The AWS Provider enables Terraform to manage AWS resources.
https://registry.terraform.io/providers/hashicorp/aws
Mozilla Public License 2.0
9.73k stars 9.09k forks source link

Running into throttling issues with many firehose deployments #11187

Open ghost opened 4 years ago

ghost commented 4 years ago

This issue was originally opened by @ppeterson-evotek as hashicorp/terraform#23598. It was migrated here as a result of the provider split. The original body of the issue is below.




* aws_kinesis_firehose_delivery_stream.kinesis_stream_es_nginx_access_admin: Error Updating Kinesis Firehose Delivery Stream: "firehose-nginx-access-admin"
InvalidArgumentException: Received ThrottlingException while describing the ElasticSearch domain.
    status code: 400, request id: fdce4a2e-c880-c71f-abaf-34dcdfeff05c
* module.elasticsearch_domain.aws_kinesis_firehose_delivery_stream.kinesis_stream_es_nginx_access_api: 1 error(s) occurred:

* aws_kinesis_firehose_delivery_stream.kinesis_stream_es_nginx_access_api: Error Updating Kinesis Firehose Delivery Stream: "firehose-nginx-access-api"
InvalidArgumentException: Received ThrottlingException while describing the ElasticSearch domain.
    status code: 400, request id: c7864577-ebed-e197-91e7-3b805a6478b5
* module.elasticsearch_domain.aws_kinesis_firehose_delivery_stream.kinesis_stream_es_nginx_access_webhooks: 1 error(s) occurred:

* aws_kinesis_firehose_delivery_stream.kinesis_stream_es_nginx_access_webhooks: Error Updating Kinesis Firehose Delivery Stream: "firehose-nginx-access-webhooks"
InvalidArgumentException: Received ThrottlingException while describing the ElasticSearch domain.
    status code: 400, request id: fbfc1ac7-827e-9a4a-ad9d-6430a3e25340
* module.elasticsearch_domain.aws_kinesis_firehose_delivery_stream.kinesis_stream_es_nginx_access_consumer-out: 1 error(s) occurred:

* aws_kinesis_firehose_delivery_stream.kinesis_stream_es_nginx_access_consumer-out: Error Updating Kinesis Firehose Delivery Stream: "firehose-nginx-access-consumer-out"
InvalidArgumentException: Received ThrottlingException while describing the ElasticSearch domain.
    status code: 400, request id: fe5a8875-9b7c-0377-a83b-f68fdcb325bc
* module.elasticsearch_domain.aws_kinesis_firehose_delivery_stream.kinesis_stream_es_nginx_error_consumer-out: 1 error(s) occurred:```

# Expected Behavior
All 60 kinesis streams should have been updated

### Actual Behavior
Api throttling stopped the deployment

### Steps to Reproduce
1. Create 60+ kinesis firehose streams in terraform
2. Apply

### Additional Context
I contacted amazon to see if there was some way to increase the throttling threshhold, but I doubt this is an option, so putting in some kind of delay on terraform's side would i guess be what i expect
justinretzolk commented 2 years ago

Hey @ppeterson-evotek 👋 Thank you for taking the time to file this issue. Given that there's been a number of AWS provider releases since you initially filed it, can you confirm whether you're still experiencing this behavior?