search

dnsimple / terraform-provider-dnsimple

Terraform DNSimple provider.
https://www.terraform.io/docs/providers/dnsimple/
Mozilla Public License 2.0
22 stars 20 forks source link

How to avoid rate limiting on large account #210

Closed sirkirby closed 5 months ago

sirkirby commented 5 months ago

Hi there. I have a rather large account, 1700 DNS Zones, that we are in the process of moving to Terraform. After 20 or so Zone imports, with a few hundred records total, I'm consistently hitting rate limits (2400/hr) when trying to plan and apply changes. Let's say I have an average of 15 records per zone. Some basic math would put us over the hourly rate limit on a single terraform plan cli command. I'm following an issue on this https://github.com/hashicorp/terraform/issues/31094, which has been open for a while as it is a complex topic. I figured I would start a conversation here, perhaps around best practices with the dnsimple provider. I can, and plan to, start splitting up my state, which could alleviate the frequency of hitting the rate limits in the short term. However, there is only so much I can do with that before it becomes impractical, as there will still likely be some state files that are quite large. Any thoughts, suggestions, or future changes that could help would be appreciated.

i'm using the lastest version of the provider, v 1.5.0 and the dnsimple_zone and dnsimple_zone_record resources.

weppos commented 5 months ago

Hi @sirkirby, thanks for your comment. We are aware that Terraform usage can become API-request intensive. There are a few options to alleviate the pressure, although it really depends on individual use cases.

For example, we provide an option that is called prefetch that works particularly well for multiple zones: https://registry.terraform.io/providers/dnsimple/dnsimple/latest/docs#prefetch

Furthermore, we have the ability to increase pagination size and/or API limit, under certain criteria and after a verification of the account.

sirkirby commented 5 months ago

Hey @weppos thanks for responding. I will certainly enable the prefetch option, thank you for that! I would like to see about increasing the rate limit as well. What do you need from me to verify the account?

weppos commented 5 months ago

Hey @weppos thanks for responding. I will certainly enable the prefetch option, thank you for that! I would like to see about increasing the rate limit as well. What do you need from me to verify the account?

Reach out to support, and we'll follow up there.