microsoft / azure-pipelines-agent

Azure Pipelines Agent 🚀
MIT License
1.7k stars 856 forks source link

[enhancement]: _sigintTimeout should be modifiable by task creator or pipeline #4364

Open steph409 opened 11 months ago

steph409 commented 11 months ago

Describe your feature request here

_sigintTimeout should be modifiable by task creator or pipeline

Hi!

Business case We are working successfully with Azure DevOps pipelines for multiple use cases. One use case is the rollout of infrastructure using terraform. Terraform has a graceful shutdown, when sigint is send. It stops to modify the resources in the cloud and updates the terraform state file, so that the configuration is consistent. After finishing the update, terraform releases the lock on the state file. When sigkill is send, the lock on the state file remains (and the state is not updated resulting in an inconsistent configuration). The next run will not work and the state lock will need to be removed manually. Terraform is quite slow, I would say it usually takes 10-30 seconds for the graceful shutdown depending on the size of the terraform configuration, with up to 1-2 minutes maybe as maximum.

Current technical implementation When a pipeline is cancelled, Azure Devops does exactly what's expected. First send a sigint, then wait for 7500ms. If the sigint has not finished by then, a sigkill is send. The 7500ms is hardcoded in the code. This limit is not enough for e.g. terraform to do a graceful shutdown. Here is an open issue from 2020 in the terraform task repository, pointing out this behavior.

Possible implementation It would be great if the hardcoded timeout would be modifiable up to a certain hard upper limit, maybe 2 minutes or so. Either by the pipeline, which could increase this value, or, if you do not want this, maybe at least task creators could specify how long their graceful shutdown takes.

Best regards, Stephi

aleksandrlevochkin commented 11 months ago

Hi @steph409, thanks for reporting! We are working on more prioritized issues at the moment but will get back to this one soon.

peruzzof commented 10 months ago

Hi, any update on this issue?

Bouke commented 4 months ago

This issue causes serious problems for us when using Terraform. We want to be able to cancel a pipeline, but it SHOULD NOT terminate (sigkill) Terraform. Just ask it to stop nicely and that's it (sigint).