search

huserben / TfsExtensions

Extensions for TFS 2015+ such as custom Widgets (require TFS 2017) and Build Tasks
MIT License
44 stars 22 forks source link

ETIMEOUT happens on scheduled jobs but not when run directly, Trigger Build Task #58

Closed OlaO closed 6 years ago

OlaO commented 6 years ago

In VSTS we have a scheduled job that runs every night. The job consists of a handful of trigger tasks. Since 5th of Mars 2018 these triggered tasks has failed with the error code "connect ETIMEOUT" and then an ip and port 443. The failure occurs in different trigger tasks every time. Sometimes it manages to complete 3, sometimes 2, sometimes none.

OlaO commented 6 years ago

It turns out there was an unanounced proxy change on the network. Setting http_proxy variable on the agent solves the issue.

JeffRausch commented 6 years ago

I'm actually seeing this same error now randomly. We have a full stack build that triggers 50+ build configurations in order and takes 45-50 minutes to complete. Most of the time it succeeds but randomly it fails with this same error. The past few days I have noticed that the VSTS console responds very slow at times, it seems like the rest api calls are timing out due to load or something. Is there any type of retry that's done or is there a way we could potentially increase the timeout for the api calls?

2018-03-16T09:30:58.0396243Z Build 4328 has not yet completed 2018-03-16T09:30:58.0396969Z Sleeping for 5 seconds... 2018-03-16T09:31:03.1238742Z Build 4328 has not yet completed 2018-03-16T09:31:03.1239464Z Sleeping for 5 seconds... 2018-03-16T09:31:29.2034739Z ##[error]connect ETIMEDOUT 13.107.6.175:443

EDIT: Opened a new issue since this is marked as closed. (https://github.com/huserben/TfsExtensions/issues/59)