logdna / terraform-provider-logdna

Terraform provider to simplify creation of views, preset alerts and other resources in LogDNA!
https://registry.terraform.io/providers/logdna/logdna/latest
MIT License
11 stars 12 forks source link

The terraform provider logdna does not handle rate limited responses #104

Open luiof opened 6 months ago

luiof commented 6 months ago

We are trying to create resources on the LogDNA provided by IBM and when the set of resources to create is huge, we get always a rate limited response.

Error: Cannot read the remote view resource
│  with module.resource_alerts_all_ghost.logdna_view.resource_containers_crash,
│  on modules/ghost-all-generic/main.tf line 47, in resource "logdna_view" "resource_containers_crash":
│  47: resource "logdna_view" "resource_containers_crash" {
│ 
│ GET https://api.eu-gb.logging.cloud.ibm.com/v1/config/view/xxxxxxx, status 429 NOT OK! {"code":"ETOOMANY","message":"Rate limit exceeded","status":"429"}

image

The problem can be skipped in the apply since we can use some delays between the resources; but in the destroy the problem is still there and the terraform state is not consistent due to the rate limited responses.

This should be handled by the provider.

jakedipity commented 3 months ago

This shouldn't be too difficult to implement. The logic in requestConfig.MakeRequest() can be extended to retry requests that fail with a 429 error code. I think adding parameters for max retries, exponential backoff, and jitter should all help with the problem described here.

Additionally those parameters should be exposed through the provider config so they can be fine-tuned.