search

geeklearningio / gl-vsts-tasks-yarn

Yarn Package Manager Visual Studio Team Services Build and Release Management extensions
MIT License
52 stars 18 forks source link

Yarn install task failing consistently with [error]connect ETIMEDOUT 104.27.154.154:443 #81

Closed moizgh closed 4 years ago

moizgh commented 4 years ago

Are you reporting a bug or a feature request ?

Description

Yarn install task is failing consistently with [error]connect ETIMEDOUT 104.27.154.154:443

Expected behavior

Yarn install task should not fail

Reproduction

Insert reproduction steps here

MANDATORY FOR BUGS: Insert debug trace

2020-04-13T19:54:45.2933122Z ##[debug]Evaluating condition for step: 'Use Yarn 1.x' 2020-04-13T19:54:45.2933645Z ##[debug]Evaluating: SucceededNode() 2020-04-13T19:54:45.2933746Z ##[debug]Evaluating SucceededNode: 2020-04-13T19:54:45.2933898Z ##[debug]=> True 2020-04-13T19:54:45.2934038Z ##[debug]Result: True 2020-04-13T19:54:45.2934168Z ##[section]Starting: Use Yarn 1.x 2020-04-13T19:54:45.3045838Z ============================================================================== 2020-04-13T19:54:45.3045907Z Task : Yarn Tool Installer 2020-04-13T19:54:45.3046132Z Description : Installs a version of Yarn 2020-04-13T19:54:45.3046176Z Version : 3.0.1999 2020-04-13T19:54:45.3046202Z Author : Geek Learning 2020-04-13T19:54:45.3046229Z Help : More Information (Version 3.0.1999) 2020-04-13T19:54:45.3046274Z ============================================================================== 2020-04-13T19:54:45.5569889Z ##[debug]agent.TempDirectory=E:_work_temp 2020-04-13T19:54:45.5647965Z ##[debug]loading inputs and endpoints 2020-04-13T19:54:45.5651728Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN 2020-04-13T19:54:45.5662381Z ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION 2020-04-13T19:54:45.5665611Z ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION 2020-04-13T19:54:45.5667371Z ##[debug]loading INPUT_ARGUMENTS 2020-04-13T19:54:45.5668994Z ##[debug]loading INPUT_CHECKLATEST 2020-04-13T19:54:45.5670231Z ##[debug]loading INPUT_INCLUDEPRERELEASE 2020-04-13T19:54:45.5671809Z ##[debug]loading INPUT_VERSIONSPEC 2020-04-13T19:54:45.5676156Z ##[debug]loading SECRET_SYSTEM_ACCESSTOKEN 2020-04-13T19:54:45.5681216Z ##[debug]loaded 8 2020-04-13T19:54:45.5694559Z ##[debug]Agent.ProxyUrl=undefined 2020-04-13T19:54:45.5694999Z ##[debug]Agent.CAInfo=undefined 2020-04-13T19:54:45.5696364Z ##[debug]Agent.ClientCert=undefined 2020-04-13T19:54:45.5696575Z ##[debug]Agent.SkipCertValidation=undefined 2020-04-13T19:54:45.5898111Z ##[debug]Agent.ProxyUrl=undefined 2020-04-13T19:54:45.5899518Z ##[debug]Agent.CAInfo=undefined 2020-04-13T19:54:45.5900468Z ##[debug]Agent.ClientCert=undefined 2020-04-13T19:54:45.5904256Z ##[debug]check path : E:_work_tasks\YarnInstaller_2ec1147b-5c33-41df-94b5-676758d87c1b\3.0.1999\node_modules\azure-pipelines-tool-lib\lib.json 2020-04-13T19:54:45.5906425Z ##[debug]adding resource file: E:_work_tasks\YarnInstaller_2ec1147b-5c33-41df-94b5-676758d87c1b\3.0.1999\node_modules\azure-pipelines-tool-lib\lib.json 2020-04-13T19:54:45.5907146Z ##[debug]system.culture=en-US 2020-04-13T19:54:45.6413679Z ##[debug]Agent.BuildDirectory=E:_work\1 2020-04-13T19:54:45.6428213Z ##[debug]versionSpec=1.x 2020-04-13T19:54:45.6430953Z ##[debug]checkLatest=false 2020-04-13T19:54:45.6432209Z ##[debug]includePrerelease=false 2020-04-13T19:54:45.6437709Z ##[debug]isExplicit: null 2020-04-13T19:54:45.6438052Z ##[debug]explicit? false 2020-04-13T19:54:45.6438133Z ##[debug]isExplicit: null 2020-04-13T19:54:45.6438279Z ##[debug]explicit? false 2020-04-13T19:54:45.6476714Z ##[debug]Agent.Version=2.164.6 2020-04-13T19:54:45.6477681Z ##[debug]Agent.ToolsDirectory=E:_work_tool 2020-04-13T19:54:45.6478356Z ##[debug]evaluating 0 versions 2020-04-13T19:54:45.6478827Z ##[debug]match not found 2020-04-13T19:54:45.6479307Z ##[debug]isExplicit: null 2020-04-13T19:54:45.6479444Z ##[debug]explicit? false 2020-04-13T19:54:45.6479542Z ##[debug]downloading: https://publicblobs.geeklearning.io/yarn/tarballsV2.json 2020-04-13T19:55:06.6731281Z ##[debug]task result: Failed 2020-04-13T19:55:06.6778158Z ##[error]connect ETIMEDOUT 104.27.154.154:443 2020-04-13T19:55:06.6787904Z ##[debug]Processed: ##vso[task.issue type=error;]connect ETIMEDOUT 104.27.154.154:443 2020-04-13T19:55:06.6788378Z ##[debug]Processed: ##vso[task.complete result=Failed;]connect ETIMEDOUT 104.27.154.154:443 2020-04-13T19:55:06.6810091Z ##[section]Finishing: Use Yarn 1.x

In order to allow us to quickly reproduce you issue please include a link to a reproduction repository

System information

VSTS or TFS:

Agent:

OS:

Yarn version: *Yarn version**

Node version: *Node version**

dedicad commented 4 years ago

It seems that I also have an error with the yarn build task. Everything was working perfectly for several weeks until Sunday. My system information are the same.

Stack Trace :

2020-04-14T04:08:50.2444442Z info There appears to be trouble with your network connection. Retrying...
2020-04-14T04:08:56.3383164Z info There appears to be trouble with your network connection. Retrying...
2020-04-14T04:09:39.3758843Z error An unexpected error occurred: "https://registry.yarnpkg.com/date-fns/-/date-fns-2.12.0.tgz: ESOCKETTIMEDOUT".

Edit

For our pipeline it was probably due to a slower connection (Azure side, or yarn/npm side, I don't know). So setting network-timeout to a higher value solved it. Here are some commands that could fix your issue.

Network timeout for yarn are also shown for a lot of errors, linked with lock file present and so on, so you might want to check other issues that you could find online. (for example : https://github.com/yarnpkg/yarn/issues/5259)

moizgh commented 4 years ago

@dedicad Thank you for the pointers. We already have network-timeout 600000 set in the .yarnrc This seems to occur on some agents quite consistently as compared to others. From the logs posted, it doesn't look like we are waiting for that long before failing.

sandorfr commented 4 years ago

hi @moizgh what's the agent spec you arw using for your pipelines?

moizgh commented 4 years ago

windows 2016 server machines

Agent.OS | Windows_NT |   |     | Agent.OSArchitecture | X64 |   |     | Agent.OSVersion | 10.0.14393 NUMBER_OF_PROCESSORS | 8 PROCESSOR_ARCHITECTURE | AMD64 |   |     | PROCESSOR_IDENTIFIER | Intel64 Family 6 Model 85 Stepping 4, GenuineIntel |   |     | PROCESSOR_LEVEL | 6 |   |     | PROCESSOR_REVISION | 5504

sandorfr commented 4 years ago

ah so you are using self hosted build agents?

sandorfr commented 4 years ago

I could not reproduce the issue so far on my own agents (VSTS or self hosted).

From the logs it actually can't download the version file, It looks like cloudflare is having some issues https://www.cloudflarestatus.com/

moizgh commented 4 years ago

Yes. We are using self hosted machines. It impacts only certain agents in the pool quite consistently. Is there retry logic we can introduce to improve the reliability?

moizgh commented 4 years ago

Added retries to the CI definition. Closing this issue for now. Will monitor progress and re-open if necessary. Thank you all for the help with the investigation.