Yarn is removing incorrect duplicated dependencies directories

[sir-gon]

Do you want to request a feature or report a bug?

bug

What is the current behavior?

Some of my dependencies have "lodash" as dependency. In a specific project, when I'm performing a "yarn install", then mi post-install fails because one post-install scripts have a require to lodash, but can't resolve as a dependency.

In my node_modules exists a lodash directory, but it doesn't have a package.json file, so... node can't resolve the require, an finally the "yarn install" fails.

When I perform a "npm install" I saw some differences in my directory structure. For example: WITH NPM: serverless is requiring lodash in post-install. With node_modules/serverless/node_modules/lodash exists and have a package.json. WITH YARN: node_modules/serverless/node_modules/lodash doesn't exists.

In both cases, node_modules/lodash exists, but without package.json inside.

In ANOTHER environment (same project, same yarn version), yarn install is working adding "lodash.once" as dependency, but now, serverless dependency is requiring jws and it can't resolve in runtime. serverless has a jws in their package.json

Then I added jws dependecy and finally works... but now in my local environment is broken again...

If the current behavior is a bug, please provide the steps to reproduce.

My package.json

{
  "name": "test",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "dependencies": {
    "babel-runtime": "^6.26.0",
    "form-data": "^2.3.1",
    "mongodb": "^3.0.2",
    "request": "^2.83.0",
    "request-promise-native": "^1.0.5"
  },
  "devDependencies": {
    "babel-core": "^6.26.0",
    "babel-loader": "^7.1.2",
    "babel-plugin-transform-runtime": "^6.23.0",
    "babel-preset-env": "^1.6.1",
    "babel-preset-es2015": "^6.24.1",
    "babel-preset-stage-0": "^6.24.1",
    "babel-preset-stage-3": "^6.24.1",
    "eslint": "^4.16.0",
    "serverless": "^1.26.0",
    "serverless-offline": "^3.16.0",
    "serverless-webpack": "^4.2.0",
    "webpack": "^3.10.0",
    "webpack-node-externals": "^1.6.0"
  }
}

then:

yarn install

Compare it with a node install (native or nodejs:6 based docker container).

What is the expected behavior?

lodash and any other duplicated dependy being installed with their package.json

Please mention your node.js, yarn and operating system version.

I'm using: docker 17.12.0-ce, build c97c6d6 node 6.12 (I can't upgrade it because a platform restriction) yarn 1.3.2

And

macOS 10.13.3 node 9.5.0 (homebrew) yarn 1.3.2 (homebrew)

[rally25rs]

Thanks for posting your package.json file contents. I tried using it and my install seems to run fine:

~/Projects/yarn-test 🐒   yarn
yarn install v1.3.2
info No lockfile found.
[1/4] 🔍  Resolving packages...
warning babel-preset-es2015@6.24.1: 🙌  Thanks for using Babel: we recommend using babel-preset-env now: please read babeljs.io/env to update!
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
success Saved lockfile.
✨  Done in 15.30s.

~/Projects/yarn-test 🐒   cat node_modules/lodash/package.json
{
  "name": "lodash",
  "version": "4.17.4",
  "description": "Lodash modular utilities.",
  "keywords": "modules, stdlib, util",
  "homepage": "https://lodash.com/",
  "repository": "lodash/lodash",
  "icon": "https://lodash.com/icon.svg",
  "license": "MIT",
  "main": "lodash.js",
  "author": "John-David Dalton <john.david.dalton@gmail.com> (http://allyoucanleet.com/)",
  "contributors": [
    "John-David Dalton <john.david.dalton@gmail.com> (http://allyoucanleet.com/)",
    "Mathias Bynens <mathias@qiwi.be> (https://mathiasbynens.be/)"
  ],
  "scripts": { "test": "echo \"See https://travis-ci.org/lodash/lodash-cli for testing details.\"" }
}

Possibly try a yarn cache clean, then delete node_modules, and re-run yarn install and see if that works?

[sir-gon]

woohaa... that works! but...

Now my suspicions are raised to the "quality" of the package cache ...

My process of continuous integration and continuous delivery rests on the premise speed (why I chose yarn), and this is critical to maintain local disk cache packages.

Now I can not completely trust this cache.

Have not you thought about eventually keeping some type of checksum per package?