search

huserben / TfsExtensions

Extensions for TFS 2015+ such as custom Widgets (require TFS 2017) and Build Tasks
MIT License
44 stars 22 forks source link

ECONNRESET issues #108

Closed geofflamrock closed 5 years ago

geofflamrock commented 5 years ago

Hi,

I've been seeing some issues when triggering builds from a release pipeline where the call fails with an ECONNRESET repeatedly. The call is being retried 5 times but each time is still failing. I noticed that the retries are very close together (around 10 - 15ms apart, so the whole retry process was done within about 300ms), and was wondering whether there was a way to have some sort of delay in-between the retries, to guard against times when DevOps is having an issue for a few seconds and the whole retry loop fits within this window?

Thanks.

huserben commented 5 years ago

Hi @geofflamrock

Thanks for your feedback. I quickly checked and it's true that we don't wait at all at the moment and just set off the next request.

I think I'll add a delay somewhere around 500ms and 1s between the retries, or what would your proposal be?

Also I'm rather busy at the moment and won't have time to make and test a change at least the next two weeks. I hope that's fine and you're not stuck in general due to this issue.

geofflamrock commented 5 years ago

Hi @huserben,

Thanks for the reply, yes some sort of random or potentially even an exponential backoff type delay (first 1s, then 2s, then 4s etc.) might be helpful. The delay could even be configurable like the setting that allows delay between triggering multiple builds if you wanted to go that far, but may not be necessary for this robustness improvement.

I'm not blocked by this issue anymore, as I believe it may also be connected to something else in the environment (it happened 4 or 5 times in a row and I don't expect I was unlucky enough to get Azure DevOps having issues at that exact time that many times in a row), and I've adjusted the pipeline structure to achieve what we wanted to do another way. I do believe a small change like this might be helpful though.

Thanks for your assistance with this.

huserben commented 5 years ago

Hi @geofflamrock

The newest version of the tasks will have the discussed handling of the wating time.

I will close the issue as I cannot test it under "real" conditions (unless you know how to force Azure DevOps to go down :-)). If you feel like it's not properly down please don't hesitate to reopen the issue.

I wanted to thank you again for reporting this problem and please feel free to open a new issue in case you find another problem, have an idea for a feature or a question.

geofflamrock commented 5 years ago

Thank you @huserben for including this feature, if we encounter any issues I'll let you know.

Thanks for making a great extension :)