timeout: resume timer on spot recovery

iterative / terraform-provider-iterative

☁️ Terraform plugin for machine learning workloads: spot instance recovery & auto-termination | AWS, GCP, Azure, Kubernetes

Apache License 2.0

288 stars 27 forks source link

Closed casperdcl closed 2 years ago

casperdcl commented 2 years ago

The timeout timer currently resets on spot interrupts.

For example, timeout=7*24*60*60 # 7days will likely never be triggered, which defeats the point IMO.

0x2b3bfa0 commented 2 years ago

Note that k8s already uses a task-scoped deadline and this issue only affects aws, az and gcp

dacbd commented 2 years ago

Could we use a timer unit that gets synced with the bucket?

casperdcl commented 2 years ago

exactly