count loop with delay feature

[fghoraishi]

Terraform Version

When I use count in my module to iterate over a list of items, it would be good to be able to put a delay timer between each iterations.

I am creating 20 subnets in IBM cloud and it cannot handle 20 requests at once.

If count had a delay parameter to pause between each iteration that would help.

Faad

Use Cases

Creating multiple resources in a loop with count. Some cloud providers do not perform well when there are too many actions at once.

Attempted Solutions

i had to break up the count module into multiple modules each with it own count. The count parameter was chunked to smaller list.

Proposal

put a delay parameters when using count in a moduile to delay between each iteration.

References

No response

[apparentlymart]

Hi @fghoraishi! Thanks for this feature request.

We typically prefer to have this sort of constraint handled automatically by the provider when that's possible. In this case, if the IBM Cloud API has a documented limit on concurrent requests or request rate then the most ideal solution would be for the IBM Cloud provider to throttle itself to stay within those limits automatically, without any special configuration on your part.

Of course that doesn't mean we can't still consider supporting this manual approach, but I think it would be worth opening a similar feature request in the repository for the IBM Cloud provider first, to see if it's feasible for the provider to handle this automatically on your behalf.

---

Notes for a future person considering implementation ideas for this request:

In Terraform's current design, there are no dependency edges whatsoever between multiple instances of the same resource, and so they always all become runnable at the same time and Terraform will act on them as fast as the global concurrency limit will allow.

Adding a delay as requested here has the prerequisite of first supporting sequential visits to the multiple instances in some sort of order. That could be achieved either by adding dependency edges to the subgraph of instances to force them into an order, or by skipping the subgraph creation altogether and literally visiting them in a regular loop.

The regular loop approach would, I think, make it easier to achieve this feature request because then that loop would have all of the context required to enforce a maximum request rate as local state, rather than having to somehow coordinate that between multiple independent graph nodes.

However, using a regular loop instead of a concurrent graph walk is a considerably different approach to the normal behavior, so the code paths would be considerably more divergent by that approach.

Maybe there's another compromise to find between those two positions.

[crw]

If you are viewing this issue and would like to indicate your interest, please use the 👍 reaction on the issue description to upvote this issue. We also welcome additional use case descriptions. Thanks!