search

yarnpkg / yarn

The 1.x line is frozen - features and bugfixes now happen on https://github.com/yarnpkg/berry
https://classic.yarnpkg.com
Other
41.44k stars 2.73k forks source link

Yarn check [--integrity [--verbose]] fail and messages aren't helpful in the slightest #2287

Open taoeffect opened 7 years ago

taoeffect commented 7 years ago

Do you want to request a feature or report a bug?

Bug.

If the current behavior is a bug, please provide the steps to reproduce.

In my repo I ran several commands, in succession: yarn check, then yarn check --integrity and then yarn check --integrity --verbose.

$ yarn check
yarn check v0.17.10
warning standard#eslint-config-standard-jsx#eslint@>=3.0.0 could be deduped from 3.12.2 to eslint@3.12.2
warning standard#eslint-config-standard#eslint@>=3.8.1 could be deduped from 3.12.2 to eslint@3.12.2
warning standard#eslint-plugin-react#eslint@^2.0.0 || ^3.0.0 could be deduped from 3.12.2 to eslint@3.12.2
warning standard#eslint-plugin-standard#eslint@>=3.0.0 could be deduped from 3.12.2 to eslint@3.12.2
warning sqlite3#node-pre-gyp#mkdirp@~0.5.1 could be deduped from 0.5.1 to mkdirp@0.5.1
warning sqlite3#node-pre-gyp#nopt@~3.0.6 could be deduped from 3.0.6 to nopt@3.0.6
warning sqlite3#node-pre-gyp#npmlog@^4.0.0 could be deduped from 4.0.1 to npmlog@4.0.1
warning sqlite3#node-pre-gyp#rc@~1.1.6 could be deduped from 1.1.6 to rc@1.1.6
warning sqlite3#node-pre-gyp#request@^2.75.0 could be deduped from 2.79.0 to request@2.79.0
warning sqlite3#node-pre-gyp#rimraf@~2.5.4 could be deduped from 2.5.4 to rimraf@2.5.4
warning sqlite3#node-pre-gyp#semver@~5.3.0 could be deduped from 5.3.0 to semver@5.3.0
warning sqlite3#node-pre-gyp#tar@~2.2.1 could be deduped from 2.2.1 to tar@2.2.1
warning sqlite3#node-pre-gyp#tar-pack@~3.3.0 could be deduped from 3.3.0 to sqlite3#tar-pack@3.3.0
error shelljs#glob is wrong version: expected 7.0.6, got 7.1.1
error es3ify#esprima-fb is wrong version: expected 3001.1.0-dev-harmony-fb, got 3001.0001.0000-dev-harmony-fb
error sass-graph#glob is wrong version: expected 7.0.6, got 7.1.1
warning fsevents#node-pre-gyp#mkdirp@~0.5.1 could be deduped from 0.5.1 to mkdirp@0.5.1
warning fsevents#node-pre-gyp#nopt@~3.0.6 could be deduped from 3.0.6 to nopt@3.0.6
error fsevents#node-pre-gyp#npmlog@^4.0.0 doesn't satisfy found match of fsevents#npmlog@3.1.2
warning fsevents#node-pre-gyp#rc@~1.1.6 could be deduped from 1.1.6 to rc@1.1.6
error fsevents#node-pre-gyp#request@^2.75.0 doesn't satisfy found match of fsevents#request@2.73.0
error fsevents#node-pre-gyp#rimraf@~2.5.4 doesn't satisfy found match of fsevents#rimraf@2.5.3
error fsevents#node-pre-gyp#semver@~5.3.0 doesn't satisfy found match of fsevents#semver@5.2.0
warning fsevents#node-pre-gyp#tar@~2.2.1 could be deduped from 2.2.1 to tar@2.2.1
warning fsevents#node-pre-gyp#tar-pack#debug@~2.2.0 could be deduped from 2.2.0 to debug@2.2.0
warning fsevents#node-pre-gyp#tar-pack#fstream@~1.0.10 could be deduped from 1.0.10 to fstream@1.0.10
warning fsevents#node-pre-gyp#tar-pack#fstream-ignore@~1.0.5 could be deduped from 1.0.5 to fstream-ignore@1.0.5
warning fsevents#node-pre-gyp#tar-pack#rimraf@~2.5.1 could be deduped from 2.5.4 to rimraf@2.5.4
warning fsevents#node-pre-gyp#tar-pack#tar@~2.2.1 could be deduped from 2.2.1 to tar@2.2.1
warning fsevents#node-pre-gyp#tar-pack#uid-number@~0.0.6 could be deduped from 0.0.6 to uid-number@0.0.6
warning fsevents#tar-pack#once#wrappy@1 could be deduped from 1.0.2 to wrappy@1.0.2
warning fsevents#tar-pack#readable-stream#buffer-shims@^1.0.0 could be deduped from 1.0.0 to buffer-shims@1.0.0
warning fsevents#tar-pack#readable-stream#core-util-is@~1.0.0 could be deduped from 1.0.2 to core-util-is@1.0.2
warning fsevents#tar-pack#readable-stream#inherits@~2.0.1 could be deduped from 2.0.3 to inherits@2.0.3
warning fsevents#tar-pack#readable-stream#process-nextick-args@~1.0.6 could be deduped from 1.0.7 to process-nextick-args@1.0.7
warning fsevents#tar-pack#readable-stream#string_decoder@~0.10.x could be deduped from 0.10.31 to string_decoder@0.10.31
warning fsevents#tar-pack#readable-stream#util-deprecate@~1.0.1 could be deduped from 1.0.2 to util-deprecate@1.0.2
info Found 30 warnings.
error Found 7 errors.

So I was like, "Wtf does that mean and how do I fix it?"

Not finding an answer to that question, I tried yarn check --integrity:

$ yarn check --integrity
yarn check v0.17.10
error Integrity hashes don't match, expected 0d2d5ed99a323c75f28f927eacdca204a7f608efc8e8c6c51fad33f05a40bc90 but got 126117d25ed334e75dc9d9e850da60dbf3a4503ef90f9a9b8547117c181db872
error Found 1 errors.
info Visit https://yarnpkg.com/en/docs/cli/check for documentation about this command.

Oh, OK. Something's broken, but I have no idea what it is or what to do about it.

Maybe yarn check --integrity --verbose will be more helpful?

$ yarn check --integrity --verbose
yarn check v0.17.10
error Integrity hashes don't match, expected 0d2d5ed99a323c75f28f927eacdca204a7f608efc8e8c6c51fad33f05a40bc90 but got 126117d25ed334e75dc9d9e850da60dbf3a4503ef90f9a9b8547117c181db872
verbose Error: Found 1 errors.
    at Object.<anonymous> (/usr/local/Cellar/yarn/0.17.10/libexec/lib/node_modules/yarn/lib/cli/commands/check.js:179:13)
    at Generator.next (<anonymous>)
    at step (/usr/local/Cellar/yarn/0.17.10/libexec/lib/node_modules/yarn/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
    at /usr/local/Cellar/yarn/0.17.10/libexec/lib/node_modules/yarn/node_modules/babel-runtime/helpers/asyncToGenerator.js:28:13
error Found 1 errors.
info Visit https://yarnpkg.com/en/docs/cli/check for documentation about this command.

😞

What is the expected behavior?

For Yarn to speak plain English and tell me what's going on and how to fix it.

  1. First, yarn check and its errors need to give some hint as to how to fix them.
  2. Secondly, yarn check --integrity should probably be done by default on every yarn command. Didn't this project say it was supposed to be "secure"? But it's not providing even the most basic level of security (checksum verification) unless a special secret command that I just stumbled upon is run. That's pretty unexpected behavior. The docs make it sound like it does integrity checks by default, but apparently it doesn't.
  3. Finally, if something goes wrong, Yarn should tell me what it actually is. I tried searching for the hashes it gave me (both of them) in the yarn.lock file and neither of them was there. WTF. How am I supposed to fix this problem at this point? Yarn should tell me exactly what to do.

Please mention your node.js, yarn and operating system version.

$ npm version
{ npm: '3.10.9',
  ares: '1.10.1-DEV',
  cldr: '30.0.2',
  http_parser: '2.7.0',
  icu: '58.1',
  modules: '51',
  node: '7.2.1',
  openssl: '1.0.2j',
  tz: '2016g',
  unicode: '9.0',
  uv: '1.10.1',
  v8: '5.4.500.44',
  zlib: '1.2.8' }

Mac OS X — 10.11.6.

Yarn v0.17.10. I would have done this with the latest version, v0.18.1, if that version was available on Homebrew, but it is not.

6zz commented 7 years ago

how do I resolve the "could be deduped..." warnings? I can't update the yarn.lock file directly right?

spacegoing commented 7 years ago

same issue here:

 warning "\u001b[2mchokidar#\u001b[22mfsevents#node-pre-gyp@^0.6.29"
 could be deduped from "0.6.32" to "node-pre-gyp@0.6.32"

Would please some body answer:

weedgrease commented 7 years ago

Would appriciate some answers to the questions asked above!

bhoule commented 7 years ago

The chokidar#fsevents#node-pre-gyp@^0.6.29 could be deduped from 0.6.33 to node-pre-gyp@0.6.33 warning is a result of node-pre-gyp being a bundled dependency of fsevents.

Ridermansb commented 7 years ago

I have an similar issue..

yarn check v0.20.3
error Integrity hashes don't match, expected "132aa146372c295e87a48cc37eec4cad3f21c1f7cff4046cadd5dd3e2b05c1be" but got "bc557edc7b8680dfbad1e2c711bd6dae3e87e057cc5f84a13f671008ed6e2515"
verbose Error: Found 1 errors.
    at MessageError (/Users/ridermansb/.nvm/versions/node/v6.7.0/lib/node_modules/yarn/lib/errors.js:8:5)
    at /Users/ridermansb/.nvm/versions/node/v6.7.0/lib/node_modules/yarn/lib/cli/commands/check.js:167:13
    at next (native)
    at step (/Users/ridermansb/.nvm/versions/node/v6.7.0/lib/node_modules/yarn/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
    at /Users/ridermansb/.nvm/versions/node/v6.7.0/lib/node_modules/yarn/node_modules/babel-runtime/helpers/asyncToGenerator.js:28:13
error Found 1 errors.
info Visit https://yarnpkg.com/en/docs/cli/check for documentation about this command

How can I fix this?

saarons commented 7 years ago

The fix I've been able to find comes from manually editing the yarn.lock file.

Let's say for example you have the following entries:

lodash-es@^4.17.3:
  version "4.17.4"
  resolved "https://registry.yarnpkg.com/lodash-es/-/lodash-es-4.17.4.tgz#dcc1d7552e150a0640073ba9cb31d70f032950e7"

lodash-es@^4.2.1:
  version "4.16.4"
  resolved "https://registry.yarnpkg.com/lodash-es/-/lodash-es-4.16.4.tgz#4dc3e2cf33a8c343028aa7f7e06d1c9697042599"

This can be manually edited down to:

lodash-es@^4.2.1, lodash-es@^4.17.3:
  version "4.17.4"
  resolved "https://registry.yarnpkg.com/lodash-es/-/lodash-es-4.17.4.tgz#dcc1d7552e150a0640073ba9cb31d70f032950e7"

Don't know how it got in this state but running yarn check again comes out clean.

taoeffect commented 7 years ago

Here's how it behaves with the latest 0.23.2 version:

screen_shot_2017-04-23_at_5 18 10_pm

W.T.F.

EDIT: created a separate issue for this related issue: #3242

Artoria2e5 commented 7 years ago

@spacegoing Why there are so many rubbish characters in the warning message?

These are ANSI color codes. What kind of terminal are you using here? Is is pre-Windows 10 cmd (old conhost)?

spacegoing commented 7 years ago

@Artoria2e5 I'm using osx's built-in terminal

bestander commented 7 years ago

This are some fair points, we are teaching Yarn proper English a little bit at a time :)

The integrity checks became more silent if they are not actionable and they do run by default when you install.

The warnings warning standard#eslint-config-standard-jsx#eslint@>=3.0.0 could be deduped from 3.12.2 to eslint@3.12.2 I think got fixed in 0.26.

But if they are not please send a PR!

Subtletree commented 7 years ago

Was seeing the same thing: yarn v0.27.5

yarn check 

warning "ember-wormhole#ember-cli-babel@^6.0.0" could be deduped from "6.8.2" to "ember-cli-babel@6.8.2"
warning "ember-ajax#ember-cli-babel@^6.0.0" could be deduped from "6.8.2" to "ember-cli-babel@6.8.2"
warning "ember-chrome-devtools#ember-cli-babel@^6.0.0" could be deduped from "6.8.2" to "ember-cli-babel@6.8.2"
warning "ember-cli-moment-shim#ember-cli-babel@^6.6.0" could be deduped from "6.8.2" to "ember-cli-babel@6.8.2"
warning "ember-cli-qunit#ember-cli-babel@^6.8.1" could be deduped from "6.8.2" to "ember-cli-babel@6.8.2"
... //many more

yarn.lock

ember-cli-babel@^6.0.0, ember-cli-babel@^6.0.0-beta.4, ember-cli-babel@^6.0.0-beta.7, ember-cli-babel@^6.1.0, ember-cli-babel@^6.3.0, ember-cli-babel@^6.4.1, ember-cli-babel@^6.6.0:
  version "6.6.0"
...
ember-cli-babel@^6.7.0:
  version "6.7.1"
...
ember-cli-babel@^6.8.0, ember-cli-babel@^6.8.2:
  version "6.8.2"
...
ember-cli-babel@^6.8.1:
  version "6.8.1"

Ended up having to recreate the lock file which fixed both problems

mjstelly commented 6 years ago

Which process is correct? I ran yarn check --integrity with this result:

yarn check v1.3.2
success Folder in sync.
✨  Done in 0.19s.

But running yarn check provides warnings:

yarn check v1.3.2
warning "sane#fsevents#node-pre-gyp@^0.6.39" could be deduped from "0.6.39" to "node-pre-gyp@0.6.39"
warning "@firebase/firestore#grpc#node-pre-gyp@^0.6.39" could be deduped from "0.6.39" to "node-pre-gyp@0.6.39"
warning "eslint-config-rallycoding#table#ajv-keywords#ajv@>=4.10.0" could be deduped from "5.5.2" to "ajv@5.5.2"
warning "eslint-rallycoding#table#ajv-keywords#ajv@>=4.10.0" could be deduped from "5.5.2" to "ajv@5.5.2"
warning "eslint-plugin-class-property#table#ajv-keywords#ajv@>=4.10.0" could be deduped from "5.5.2" to "ajv@5.5.2"
info Found 5 warnings.
success Folder in sync.
✨  Done in 3.90s.

Which one is to be believed?

bestander commented 6 years ago

—integrity does a light check that verifies that no one touched node_modules after yarn installed it.

Regular check runs Yarn’s resolution mechanism that checks that the installation tree matches yarn.lock. The warnings indicate that something is not optimal but installation is still correct

On Thu, Jan 18, 2018 at 5:33 AM Michael Stelly notifications@github.com wrote:

Which process is correct? I ran yarn check --integrity with this result:

yarn check v1.3.2 success Folder in sync. ✨ Done in 0.19s.

But running yarn check provides warnings:

yarn check v1.3.2 warning "sane#fsevents#node-pre-gyp@^0.6.39" could be deduped from "0.6.39" to "node-pre-gyp@0.6.39" warning "@firebase/firestore#grpc#node-pre-gyp@^0.6.39" could be deduped from "0.6.39" to "node-pre-gyp@0.6.39" warning "eslint-config-rallycoding#table#ajv-keywords#ajv@>=4.10.0" could be deduped from "5.5.2" to "ajv@5.5.2" warning "eslint-rallycoding#table#ajv-keywords#ajv@>=4.10.0" could be deduped from "5.5.2" to "ajv@5.5.2" warning "eslint-plugin-class-property#table#ajv-keywords#ajv@>=4.10.0" could be deduped from "5.5.2" to "ajv@5.5.2" info Found 5 warnings. success Folder in sync. ✨ Done in 3.90s.

Which one is to be believed?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/yarnpkg/yarn/issues/2287#issuecomment-358518614, or mute the thread https://github.com/notifications/unsubscribe-auth/ACBdWPTh_-WBrR0n66gXuV34lIo7pXKpks5tLq2DgaJpZM4LQS0w .

robbie-hinman commented 6 years ago

So... yarn check provides 13 warnings of packages that 'could be deduped' and I run yarn dedupe and am told yarn install will already dedupe and then I run yarn install and then it says I'm good and then run yarn check and I get 13 warnings... and yarn.lock says not to manually edit the file.

If it was just warnings I wouldn't be so concerned but our app recently and mysteriously broke and the only culprit is a ton of changes to a checked in yarn.lock file...

bestander commented 6 years ago

Often dedupe warnings are not really actionable by end users because this is in control of package authors. @robbie-hinman, my bet is that your break is not linked to the duplicate warnings you see. Yarn should clean this up and warn only on actionable cases, unfortunately the core team is busy with so many things it will take a lot of time till we get to fixing this. Help from someone from the community will be very much appreciated

halo commented 6 years ago

I noticed that the yarn check output posted here is usually somewhat helpful:

In my case it doesn't give me any hint as to what went wrong. I just run yarn install and then it works again for an hour or so until it breaks again with the following output 🤪

$ yarn check --integrity --verbose
yarn check v1.3.2
[...]
verbose 0.475 Checking for configuration file "/Users/.yarnrc".
verbose 0.477 current time: 2018-02-08T10:41:55.530Z
warning Integrity check: Flags don't match
error Integrity check failed
verbose 0.601 Error: Found 1 errors.
    at /path/to/homebrew/Cellar/yarn/1.3.2/libexec/lib/cli.js:57476:13
    at Generator.next (<anonymous>)
    at step (/path/to/homebrew/Cellar/yarn/1.3.2/libexec/lib/cli.js:92:30)
    at /path/to/homebrew/Cellar/yarn/1.3.2/libexec/lib/cli.js:103:13
    at <anonymous>
error Found 1 errors.
info Visit https://yarnpkg.com/en/docs/cli/check for documentation about this command.

Of course I hope this is not related to my setup but I don't think I do anything special. Just a normal Rails app using yarn.

Just to be clear: I'm not complaining, I'm just adding my output because it appears to be unique. The yarn team are heroes altogether!

bestander commented 6 years ago

Thanks for feedback Here is the error https://github.com/yarnpkg/yarn/blob/f69cdda63ab6f36bdd21c6f4ab838e7b6200b1d6/__tests__/commands/check.js#L213, here is the text https://github.com/yarnpkg/yarn/blob/1b4b318355115cff5e58f128c53709b37b6ed28e/src/reporters/lang/en.js#L360

A PR to fix this would make Yarn even better.

kaiyoma commented 6 years ago

Haven't seen any recent activity here, but wanted to report that we have been running into this issue for quite a while now. In our project, yarn check spits out anywhere between 50 and 100 warnings that look like this:

...
warning "jest-runtime#babel-core#lodash@^4.17.4" could be deduped from "4.17.10" to "lodash@4.17.10"
warning "jest-snapshot#jest-diff#chalk@^2.0.1" could be deduped from "2.4.1" to "chalk@2.4.1"
warning "jest-config#jest-jasmine2#chalk@^2.0.1" could be deduped from "2.4.1" to "chalk@2.4.1"
...

There's no explanation of what's wrong (if anything) or what to do. Instead, legitimate warnings/errors are obfuscated by a deluge of output that isn't helpful and can't be suppressed.

simonbuchan commented 5 years ago

I've noticed file:../some-dir and link: deps will cause the "could be deduped" errors, presumably because in the first case, yarn incorrectly bundles the target dir's node_modules, and in the second because check doesn't know what to do with symlinks (probably just ignore them, right?)

If I get time to verify these soon I'll open bugs for them, but for now this might let people know what's up.