search

yarnpkg / yarn

The 1.x line is frozen - features and bugfixes now happen on https://github.com/yarnpkg/berry
https://classic.yarnpkg.com
Other
41.41k stars 2.72k forks source link

Improve CI compatibility - Add yarn version to lock file #2847

Closed christopherstott closed 7 years ago

christopherstott commented 7 years ago

Referring to : https://github.com/yarnpkg/yarn/pull/2779

@bestander - I’d like to add some more context to this and get this reconsidered.

This change will be valuable for any team that uses a CI that does any kind of inference. The CI service can guarantee the correct version of yarn is used for the build.

Without this it places a burden of yarn version management onto teams that use CI. Users often forget to update CI to match their local environment, and spin frustrated because the error messages for incompatible versions can be very cryptic.

For a CI service like ours, that ends up burning a lot of support time too.

This has happened twice in the last two weeks now:

Originally in buddybuild, we were always updating to the latest yarn version for a build. About a week and a half ago a new version of yarn came out that prompted for user input in a particular circumstance. That caused builds to hang.

These hangs were affecting customers who hasn’t yet upgraded yarn locally.

When that happened we rolled back and pinned to the previous stable version 0.18.1.

It now looks like that version of yarn isn’t capable of correctly installing yarn.locks generated with the latest stable version.

We had a flurry of support requests the last few days with customers confused by yarn errors like these:

error An unexpected error occurred: “ENOENT: no such file or directory, open ‘/usr/local/lib/node_modules/yarn/node_modules/har-validator/lib/index.js’”.

The latest yarn successfully installs. Between these versions the ‘yarn lockfile’ version wasn’t bumped.

To me, it seems there are two solutions: 1) Bake the yarn version into the yarn.lock 2) Require the user to manually update their CI

Among the two big iOS package managers cocoapods & carthage, cocoapods takes the approach of baking the cocoapods version into the Podfile.lock, whereas carthage doesn’t put its version into Cartfile.resolved.

Within buddybuild cocoapods has 10x the usage of carthage, however carthage has 5x rate of failures. Many of these are due to the wrong version.

Baking in the version to yarn.lock will also help teams who want to make sure they’re in sync with yarn versions. Merge conflicts will only occur with teams with folks using different versions - and should be simple to resolve. Anecdotally, I’ve never heard of a similar complaint with this behaviour in cocoapods.

If the conflicts becomes a problem, a setting to disable this behaviour would solve that.

As the carthage case shows, requiring the user to manually update their CI will lead to more fragility.

reversefold commented 7 years ago

We're having an issue today with the same error message:

00:00:12.133 $ yarn install 
00:00:12.647 yarn install v0.19.1
00:00:13.367 [1/4] Resolving packages...
00:00:15.281 error An unexpected error occurred: "ENOENT: no such file or directory, open '/home/builder/tools/jenkins.plugins.nodejs.tools.NodeJSInstallation/NodeJS_7.5.0_with_yarn_0.19.1/lib/node_modules/yarn/node_modules/har-validator/lib/index.js'".

This is with Jenkins and the NodeJS plugin doing our tool installation. The issue appears to only affect some of our build slaves and not others. So far we can't find any difference between the systems or their NodeJS installations, all have the same version of NodeJS and the same global modules installed.

bestander commented 7 years ago

This is a community project and everything is open to discussion, thanks for bringing this up.

  1. How would a version 0.18.1 in yarn.lock file prevent the error you describe?

  2. If you regenerate yarn.lock with the latest version does the error go away?

It looks like there is an error in Yarn that broke and we just need to fix it.

bestander commented 7 years ago

If 2 is true, let's raise it as a separate issue and fix it in Yarn.

DanielJoyce commented 7 years ago

Yeah. This just horked our build too. So I locked the yarn version.

DanielJoyce commented 7 years ago

@reversefold Installing an older yarn version, which I was using on the dev box, fixed the issue for us.

yarn@0.16.1 works

The first build to break due to this was on March 6 2017

Here is the log from the build that broke

Running with gitlab-ci-multi-runner 1.9.2 (ade6572)
Using Shell executor...
Running on gitlab2...
Fetching changes...
Removing node_modules/
Removing npm-debug.log
Removing public/ts/App/App.js
Removing public/ts/App/common.js
Removing public/ts/App/index.js
Removing public/ts/BaseModal/BaseModalModel.js
Removing public/ts/BaseModal/common.js
Removing public/ts/BaseModal/index.js
Removing public/ts/NotesModal/NotesModel.js
Removing public/ts/NotesModal/common.js
Removing public/ts/NotesModal/index.js
Removing public/ts/POCModal/POCModalModel.js
Removing public/ts/POCModal/common.js
Removing public/ts/POCModal/index.js
Removing public/ts/VulnRemModal/VulnRemModalModel.js
Removing public/ts/VulnRemModal/common.js
Removing public/ts/VulnRemModal/index.js
Removing public/ts/main.js
HEAD is now at 3191a36 Switch to observable bindings instead of events as Knockout works best with that model
From https://gitlab.lightson.net/vizualiiz/tmo_vuln
   3191a36..c22aa80  new-features -> origin/new-features
Checking out c22aa802 as new-features...
$ export NVM_DIR="/home/gitlab-runner/.nvm"
$ [ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"
$ source setup_env.sh
Setting up TMO Vuln development env
bash: no job control in this shell
[tmo_vuln][gitlab-runner@gitlab2 tmo_vuln]$ exit
$ nvm install 6.6.0
v6.6.0 is already installed.
Now using node v6.6.0 (npm v3.10.3)
$ nvm alias default 6.6.0
default -> 6.6.0 (-> v6.6.0 *)
$ echo "NPM VERSION: $(npm --version)"
NPM VERSION: 3.10.3
$ echo "NODE VERSION: $(node --version)"
NODE VERSION: v6.6.0
$ git clean -fxd
$ npm install -g yarn
/home/gitlab-runner/.nvm/versions/node/v6.6.0/bin/yarn -> /home/gitlab-runner/.nvm/versions/node/v6.6.0/lib/node_modules/yarn/bin/yarn.js
/home/gitlab-runner/.nvm/versions/node/v6.6.0/bin/yarnpkg -> /home/gitlab-runner/.nvm/versions/node/v6.6.0/lib/node_modules/yarn/bin/yarn.js
- generate-function@2.0.0 node_modules/yarn/node_modules/generate-function
- is-property@1.0.2 node_modules/yarn/node_modules/is-property
- generate-object-property@1.2.0 node_modules/yarn/node_modules/generate-object-property
- jsonpointer@4.0.1 node_modules/yarn/node_modules/jsonpointer
- is-my-json-valid@2.16.0 node_modules/yarn/node_modules/is-my-json-valid
- pinkie@2.0.4 node_modules/yarn/node_modules/pinkie
- pinkie-promise@2.0.1 node_modules/yarn/node_modules/pinkie-promise
/home/gitlab-runner/.nvm/versions/node/v6.6.0/lib
└─┬ yarn@0.21.3 
  ├── inquirer@3.0.6 
  └─┬ request@2.80.0 
    ├── caseless@0.12.0 
    ├─┬ har-validator@4.2.1 
    │ ├─┬ ajv@4.11.4 
    │ │ ├── co@4.6.0 
    │ │ └─┬ json-stable-stringify@1.0.1 
    │ │   └── jsonify@0.0.0 
    │ └── har-schema@1.0.5 
    ├── performance-now@0.2.0 
    └── qs@6.3.2 

$ export PATH=$PATH:`npm bin`
$ echo "YARN VERSION: $(yarn --version)"
YARN VERSION: 0.21.3
$ yarn install
yarn install v0.21.3
warning No license field
[1/4] Resolving packages...
[2/4] Fetching packages...
error An unexpected error occurred: "ENOENT: no such file or directory, open '/home/gitlab-runner/.nvm/versions/node/v6.6.0/lib/node_modules/yarn/node_modules/har-validator/lib/index.js'".
info If you think this is a bug, please open a bug report with the information provided in "/home/gitlab-runner/builds/e632e3cb/0/vizualiiz/tmo_vuln/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
ERROR: Build failed: exit status 1

The last build we have that didn't break was Feb 23, ( code is built only on checkin, and I didn't push for a while )

Here is what yarn version was pulled for that build:

Running with gitlab-ci-multi-runner 1.9.2 (ade6572)
Using Shell executor...
Running on gitlab2...
Fetching changes...
Removing node_modules/
Removing public/js/
HEAD is now at f3cfb82 Further expanded on readme
From https://gitlab.lightson.net/vizualiiz/tmo_vuln
   f3cfb82..52a056b  master     -> origin/master
Checking out 52a056b1 as master...
$ export NVM_DIR="/home/gitlab-runner/.nvm"
$ [ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"
$ source setup_env.sh
Setting up TMO Vuln development env
bash: no job control in this shell
[tmo_vuln][gitlab-runner@gitlab2 tmo_vuln]$ exit
$ nvm install 6.6.0
v6.6.0 is already installed.
Now using node v6.6.0 (npm v3.10.3)
$ nvm alias default 6.6.0
default -> 6.6.0 (-> v6.6.0 *)
$ echo "NPM VERSION: $(npm --version)"
NPM VERSION: 3.10.3
$ echo "NODE VERSION: $(node --version)"
NODE VERSION: v6.6.0
$ git clean -fxd
$ npm install -g yarn
/home/gitlab-runner/.nvm/versions/node/v6.6.0/bin/yarn -> /home/gitlab-runner/.nvm/versions/node/v6.6.0/lib/node_modules/yarn/bin/yarn.js
/home/gitlab-runner/.nvm/versions/node/v6.6.0/bin/yarnpkg -> /home/gitlab-runner/.nvm/versions/node/v6.6.0/lib/node_modules/yarn/bin/yarn.js

> spawn-sync@1.0.15 postinstall /home/gitlab-runner/.nvm/versions/node/v6.6.0/lib/node_modules/yarn/node_modules/spawn-sync
> node postinstall

- is-fullwidth-code-point@1.0.0 node_modules/yarn/node_modules/gauge/node_modules/is-fullwidth-code-point
- string-width@1.0.2 node_modules/yarn/node_modules/gauge/node_modules/string-width
- mimic-fn@1.1.0 node_modules/yarn/node_modules/mimic-fn
- is-fullwidth-code-point@1.0.0 node_modules/yarn/node_modules/wide-align/node_modules/is-fullwidth-code-point
- string-width@1.0.2 node_modules/yarn/node_modules/wide-align/node_modules/string-width
/home/gitlab-runner/.nvm/versions/node/v6.6.0/lib
└─┬ yarn@0.18.2 
  ├── detect-indent@4.0.0 
  ├─┬ inquirer@1.2.3 
  │ ├─┬ cli-cursor@1.0.2 
  │ │ └─┬ restore-cursor@1.0.1 
  │ │   ├── exit-hook@1.1.1 
  │ │   └── onetime@1.1.0 
  │ ├─┬ external-editor@1.1.1 
  │ │ ├─┬ spawn-sync@1.0.15 
  │ │ │ ├─┬ concat-stream@1.6.0 
  │ │ │ │ └── typedarray@0.0.6 
  │ │ │ └── os-shim@0.1.3 
  │ │ └── tmp@0.0.29 
  │ ├── figures@1.7.0 
  │ ├── mute-stream@0.0.6 
  │ └─┬ string-width@1.0.2 
  │   └── is-fullwidth-code-point@1.0.0 
  ├─┬ proper-lockfile@1.2.0 
  │ └── err-code@1.1.1 
  ├─┬ repeating@2.0.1 
  │ └── is-finite@1.0.2 
  └─┬ strip-bom@2.0.0 
    └── is-utf8@0.2.1 

$ export PATH=$PATH:`npm bin`
$ echo "YARN VERSION: $(yarn --version)"
YARN VERSION: 0.18.2
$ yarn install

I suspect 0.21.3 has a broken depedency somewhere. Did a yarn dev forget to check in their yarn.lock file? ;)

reversefold commented 7 years ago

What is odd for us is that we're using yarn 0.19.1 on all of our build slaves and some of them are consistently failing with this error while some are not. We can't find any appreciable difference between the machines and we have verified that the same nodeJS install and global packages are on both working and non-working machines. Even copying installs from one machine to the other didn't help. The timing is the same as what @christopherstott seems to be seeing, though. We had builds working fine on all machines as of Friday the 3rd and on Monday the 6th some machines started failing.

jasonmit commented 7 years ago

I was also seeing this, is everyone on this thread installing yarn through npm?

reversefold commented 7 years ago

I can confirm that we are installing yarn via npm, yes. We've also tested installing a fresh version on a new machine (docker container) and that also worked fine.

jasonmit commented 7 years ago

What I ended up doing was ensuring I cleared the npm cache (npm cache clean) before installing yarn before each test run. This temporarily resolved the issue for me.

DanielJoyce commented 7 years ago

We used npm to install yarn too.

reversefold commented 7 years ago

I believe I have figured out how to reproduce this issue on our environment. In testing various solutions I happened upon a fix: setting HOME to the build's workspace (and the job wiping out its workspace before every build). Once I did this the error went away.

Once I reverted the change to override HOME I tracked the problem down to ~/.cache. Once I removed that the error also went away.

Lastly I tracked down when the builds had started having these ENOENT errors and it appears that if a build is aborted by jenkins while yarn is installing things that it leaves the ~/.cache in some kind of a bad state. Here is the last output from an aborted build that may have contributed to the problem:

$ yarn install 
yarn install v0.19.1
[1/4] Resolving packages...
[2/4] Fetching packages...
warning fsevents@1.0.17: The platform "linux" is incompatible with this module.
info "fsevents@1.0.17" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
Build was aborted
Aborted by USER
Archiving artifacts
[SHN-JDK8] $ sh -e /shn/builder/tools/hudson.model.JDK/SHN-JDK8/hudson8188784432097936548.sh
using /usr/java/1.8
Started calculate disk usage of build
Finished Calculation of disk usage of build in 0 seconds
Started calculate disk usage of workspace
error /shn/builder/workspace/test-yarn/node_modules/phantomjs-prebuilt: Command failed.
Exit code: 1
Command: sh
Arguments: -c node install.js
Directory: /shn/builder/workspace/test-yarn/node_modules/phantomjs-prebuilt
Output:
PhantomJS not found on PATH
Download already available at /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
Verified checksum of previously downloaded file
Extracting tar contents (via spawned process)
Error extracting archive
Phantom installation failed { Error: Command failed: tar jxf /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now

    at ChildProcess.exithandler (child_process.js:211:12)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:192:7)
    at maybeClose (internal/child_process.js:890:16)
    at Socket.<anonymous> (internal/child_process.js:334:11)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:189:7)
    at Pipe._handle.close [as _onclose] (net.js:501:12)
  killed: false,
  code: 2,
  signal: null,
  cmd: 'tar jxf /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2' } Error: Command failed: tar jxf /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now

    at ChildProcess.exithandler (child_process.js:211:12)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:192:7)
    at maybeClose (internal/child_process.js:890:16)
    at Socket.<anonymous> (internal/child_process.js:334:11)
    at emitOne (events.js:96:13)
    at Socket.emit (events.js:189:7)
    at Pipe._handle.close [as _onclose] (net.js:501:12)
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
vinnymac commented 7 years ago

Interestingly I am not seeing this in my Travis CI when using node 7.

In node 6 this issue does occur however. har-validator says to look for the promise file in the lib folder as its main, but instead yarn thinks it needs to look for the index file. If someone understands why this is specific to node 6, please let me know.

Below you can find what I added to travis.yml to get around the issue temporarily. This works on node 6 and 7 with yarn 0.21.3

install:
  # BEGIN PATCH for har-validator
  - export BACK_TO_WORKSPACE=$(pwd)
  - cd $(npm config get prefix)/lib/node_modules/yarn
  - npm install har-validator
  # Makes node 7 work again
  - mkdir $(npm config get prefix)/lib/node_modules/yarn/node_modules/har-validator/lib/node4
  - ln -s $(npm config get prefix)/lib/node_modules/yarn/node_modules/har-validator/lib/promise.js $(npm config get prefix)/lib/node_modules/yarn/node_modules/har-validator/lib/node4/promise.js
  - ln -s $(npm config get prefix)/lib/node_modules/yarn/node_modules/har-validator/lib/error.js $(npm config get prefix)/lib/node_modules/yarn/node_modules/har-validator/lib/node4/error.js
  - ln -s $(npm config get prefix)/lib/node_modules/yarn/node_modules/har-validator/lib/async.js $(npm config get prefix)/lib/node_modules/yarn/node_modules/har-validator/lib/node4/async.js
  # Makes node 6 work
  - ln -s $(npm config get prefix)/lib/node_modules/yarn/node_modules/har-validator/lib/promise.js $(npm config get prefix)/lib/node_modules/yarn/node_modules/har-validator/lib/index.js
  - cd $BACK_TO_WORKSPACE
  # END PATCH for har-validator

I also came across this https://github.com/timoxley/til/blob/master/javascript/yarn-enoent.md

orta commented 7 years ago

I'd also like to see this, as I'm not 100% certain all of the developers working on a project are using the same version of yarn as each other without a feature like this. 👍

bestander commented 7 years ago

Yeah, I think we should add it.

On 5 May 2017 at 14:32, Orta notifications@github.com wrote:

I'd also like to see this, as I'm not 100% certain all of the developers working on a project are using the same version of yarn as each other without a feature like this. 👍

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/yarnpkg/yarn/issues/2847#issuecomment-299465404, or mute the thread https://github.com/notifications/unsubscribe-auth/ACBdWKAwIXH-ZXrUwOCeAyOjS9zRdth6ks5r2yT1gaJpZM4MUnTe .

bestander commented 7 years ago

Done in 0.25

borekb commented 7 years ago

So how does this behave in 0.25, exactly? I've started seeing a lot of yarn.lock diffs on this (I am on 0.25 beta, other team members aren't yet). If this is in yarn.lock:

# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1
# yarn v0.25.3
# node v7.9.0

what will happen if a colleagues has 0.25.1? Will the installation be aborted and he needs to upgrade? What if he's on a newer version like 0.26?

Thanks.

borekb commented 7 years ago

Possibly related: https://github.com/yarnpkg/yarn/pull/2779#issuecomment-282787540