destroy seems single threaded?

[jstrachan]

I've not had chance to check the code yet, but it looks like if you try do delete a number of Terraform resources at the same time the operator works in a single threaded way; it starts a single destroy Job - waits for this job to complete before starting another one. This can be very slow. e.g. on AWS it can take 10 minutes or longer to destroy a single Terraform.

I wonder if we could make things more parallel so we can create multiple Terraform destroy jobs in run parallel (like the apply jobs do)?

I wonder if this is by design or just an optimisation we could make?

We're using the terraform operator for spinning up multiple clusters on multiple pipelines on each Pull Request; so often have lots of Terraform resources to tear down and its currently pretty slow (which leaves lots of resources around unused and it looks like when destroying the operator no longer creates any new Terraform either).

[isaaguilar]

Yep this should be a simple change... but every time someone says that on GH it ends up not being simple. 😓 I'll take a stab real quick.

[isaaguilar]

I might simply take out the for loop that checks if destroy is complete but I'm sure there's a much nicer solution. Will come back to this later.