search

timohirt / terraform-provider-hetznerdns

Terraform provider for Hetzner DNS
Mozilla Public License 2.0
109 stars 21 forks source link

Hitting rate limit (without hitting it) #50

Closed eugenkoenig closed 1 year ago

eugenkoenig commented 1 year ago

We manage our whole DNS infrastructure with this plugin and now started getting Error getting record with id <id>: Error getting Record. HTTP status 429 unhandled for every resource. The Hetzner docs say it's limited to 3600 requests per hour, however bursts are allowed. AFAIK, running terraform plan is a burst request. We currently manage 841 resources (entries) over 48 domains and it failed already on the first try.

It was working for us until last month, we didn't change anything so far, just added two additional records, but it's failing on plan now. We are on plugin version 2.0.0.

Someone aware, was there a change on Hetzner side? Or maybe someone experienced same issue? Is it even possible to run terraform "slower" to not to hit the rate limits?

dabo-devconsole commented 1 year ago

Just some ideas... The Hetzner docs are also talking about some response headers, for getting insight about existing rate-limits. Did you tried it in debug mode (export TF_LOG=DEBUG)? Maybe terraform plan -parallelism=1 can also be worth a shot. Don't know how complex it would be to use different Auth-API-Tokens for each domain, as rate limits should be counted per Auth-API-Token, right? When I correctly understand this provider, it retrieves each record individually. There is another API request possible for getting all records in one call(GET /records). But therefore the provider has to be altered I think.

eugenkoenig commented 1 year ago

Sorry for the late reply. We asked Hetzner about the issue last week and just got a response, that they could increase the rate limit. We'll try to go with a higher limit and see if it helps. That would be the most appropriate solution for managing a lot of records without the need to adapt the provider. As for the debugging, I'll enable debugging in the CI if I have a minute and see what it tells. Thanks for the hint!

eugenkoenig commented 1 year ago

Hetzner responded and said that they increased rate limit for us to 1k req/minute. Now it works for us.

I guess there is nothing that has to be fixed on the provider, so I'll close the issue.

For everyone who manages a lot of records and gets the same problem: just open a ticket for Hetzner and ask them to increase the rate limit.