iterative / terraform-provider-iterative

☁️ Terraform plugin for machine learning workloads: spot instance recovery & auto-termination | AWS, GCP, Azure, Kubernetes
https://registry.terraform.io/providers/iterative/iterative/latest/docs
Apache License 2.0
290 stars 27 forks source link

`Reading` messages repeated five times on `destroy` and thrice on other operations #487

Closed 0x2b3bfa0 closed 2 years ago

0x2b3bfa0 commented 2 years ago

The Read() method is being invoked several times, both from Terraform and from the task internal planner. 🟦 ☸️

See also

casperdcl commented 2 years ago

Would make users think we're stuck in an infinite loop.

Need to at least say e.g.

Reading [1/3]
[... 9 lines of output ...]
Reading [2/3]
[... 9 repeated lines ...]
Reading [3/3]
[... 9 repeated lines ...]

the increasing numbers (1/3, 2/3, 3/3) reassure it's not infinite.

0x2b3bfa0 commented 2 years ago

Regretfully, the proposed fix is not possible in the realms of structural sanity.

The way we use Terraform implies that some of the Read() operations are performed by Terraform Core and others are performed by the task internal planner. 🟦 ☸️ Neither of them can cache operations performed by the other, nor knows if a redundant read was triggered before, nor whether a redundant read will be triggered afterwards.

casperdcl commented 2 years ago

:scream:

0x2b3bfa0 commented 2 years ago

What about the following?

Before

TPI [INFO] [1/12] Reading DefaultNetwork...
TPI [INFO] [2/12] Reading Image...
TPI [INFO] [3/12] Reading Bucket...
TPI [INFO] [4/12] Reading Credentials...
TPI [INFO] [5/12] Reading FirewallInternalEgress...
TPI [INFO] [6/12] Reading FirewallInternalIngress...
TPI [INFO] [7/12] Reading FirewallExternalEgress...
TPI [INFO] [8/12] Reading FirewallExternalIngress...
TPI [INFO] [9/12] Reading FirewallDenyEgress...
TPI [INFO] [10/12] Reading FirewallDenyIngress...
TPI [INFO] [11/12] Reading InstanceTemplate...
TPI [INFO] [12/12] Reading InstanceGroupManager...
TPI [INFO] [1/12] Reading DefaultNetwork...
TPI [INFO] [2/12] Reading Image...
TPI [INFO] [3/12] Reading Bucket...
TPI [INFO] [4/12] Reading Credentials...
TPI [INFO] [5/12] Reading FirewallInternalEgress...
TPI [INFO] [6/12] Reading FirewallInternalIngress...
TPI [INFO] [7/12] Reading FirewallExternalEgress...
TPI [INFO] [8/12] Reading FirewallExternalIngress...
TPI [INFO] [9/12] Reading FirewallDenyEgress...
TPI [INFO] [10/12] Reading FirewallDenyIngress...
TPI [INFO] [11/12] Reading InstanceTemplate...
TPI [INFO] [12/12] Reading InstanceGroupManager...
TPI [INFO] [1/12] Reading DefaultNetwork...
TPI [INFO] [2/12] Reading Image...
TPI [INFO] [3/12] Reading Bucket...
TPI [INFO] [4/12] Reading Credentials...
TPI [INFO] [5/12] Reading FirewallInternalEgress...
TPI [INFO] [6/12] Reading FirewallInternalIngress...
TPI [INFO] [7/12] Reading FirewallExternalEgress...
TPI [INFO] [8/12] Reading FirewallExternalIngress...
TPI [INFO] [9/12] Reading FirewallDenyEgress...
TPI [INFO] [10/12] Reading FirewallDenyIngress...
TPI [INFO] [11/12] Reading InstanceTemplate...
TPI [INFO] [12/12] Reading InstanceGroupManager...

After

TPI [INFO] Reading... (this may happen several times)
TPI [INFO] Reading... (this may happen several times)
TPI [INFO] Reading... (this may happen several times)
dacbd commented 2 years ago

As in there are just 3 lines? or (this may happen several times) appears on every line? I would prefer it as its own single INFO line that to the effect of "reads may occur multiple times" or your "this may happen several times"

casperdcl commented 2 years ago

would prefer the more verbose option (which would downgrade this from p0 to p1)

0x2b3bfa0 commented 2 years ago

As in there are just 3 lines?

Yes, just suppress (or move to DEBUG level) all the per–step minutiae, and just print a single line for each call to Read()

0x2b3bfa0 commented 2 years ago

would prefer the more verbose option

“Yes, that one” — can you please put an example? I'm a bit lost unless we assign numbers to the proposed alternatives. 🐊

casperdcl commented 2 years ago
casperdcl commented 2 years ago

is it possible in future to optimise this (so unnecessary reads are circumvented)?

0x2b3bfa0 commented 2 years ago