TypeError: Cannot read properties of undefined (reading 'filter') forces frequent "nx reset" requirements

[dep]

Current Behavior

Currently, quite often, when I run a command like this one:

npx lerna run storybook --stream --scope=@invoca/$*

or

npx lerna run test:changed --stream --parallel --scope=@invoca/$*

Among many others...

I'll get an error related to nx:

node_modules/lerna/node_modules/nx/src/hasher/hasher.js:531
    return files.filter((f) => {
                 ^

TypeError: Cannot read properties of undefined (reading 'filter')
    at filterUsingGlobPatterns (/Users/rkearns/invoca/Titan/node_modules/lerna/node_modules/nx/src/hasher/hasher.js:531:18)
    at TaskHasher.<anonymous>

Doing an nx reset fixes this 100% of the time, but it happens so frequently, we basically have to bake nx reset into all of our Make commands to keep the error at bay. Why does this happen?

Expected Behavior

Shouldn't need to do so many nx reset

GitHub Repo

No response

Steps to Reproduce

1. Run a top-level command
2. See error

Nx Report

>  NX   Report complete - copy this into the issue template

   Node   : 18.16.1
   OS     : darwin-x64
   npm    : 9.5.1

   nx            : 16.10.0
   lerna         : 6.6.1
   @nrwl/devkit  : 15.9.4
   @nrwl/tao     : 16.10.0
   typescript    : 5.2.2
   ---------------------------------------
   The following packages should match the installed version of nx
     - @nrwl/devkit@15.9.4

   To fix this, run `nx migrate nx@16.10.0`

Failure Logs

node_modules/lerna/node_modules/nx/src/hasher/hasher.js:531
    return files.filter((f) => {
                 ^

TypeError: Cannot read properties of undefined (reading 'filter')
    at filterUsingGlobPatterns (/Users/rkearns/invoca/Titan/node_modules/lerna/node_modules/nx/src/hasher/hasher.js:531:18)
    at TaskHasher.<anonymous>

Package Manager Version

npm 9.5.1

Operating System

• [X] macOS
• [ ] Linux
• [ ] Windows
• [ ] Other (Please specify)

Additional Information

No response

[JamesHenry]

@dep Please can we try and figure out why you have that very old copy of @nrwl/devkit in the mix

Maybe try npm ls @nrwl/devkit

Additionally, please provide the output of npx lerna info for refernce

[dep]

npm ls @nrwl/devkit:

└─┬ lerna@6.6.1
  ├─┬ @lerna/legacy-package-management@6.6.1
  │ └── @nrwl/devkit@15.9.4 deduped
  └── @nrwl/devkit@15.9.4

npx lerna info

lerna notice cli v6.6.1
lerna info versioning independent

 Environment info:

  System:
    OS: macOS 14.0
    CPU: (10) x64 Apple M1 Pro
  Binaries:
    Node: 18.16.1 - ~/.nvm/versions/node/v18.16.1/bin/node
    Yarn: 1.22.19 - /usr/local/bin/yarn
    npm: 9.5.1 - ~/.nvm/versions/node/v18.16.1/bin/npm
  Utilities:
    Git: 2.39.3 - /usr/bin/git
  npmPackages:
    lerna: 6.6.1 => 6.6.1

Thanks for your help!

[dep]

Also this is my package.json

{
  "name": "foo",
  "private": true,
  "version": "0.0.0",
  "description": "Foo mono repo",
  "repository": {
    "type": "git",
    "url": "git+https://github.com/invoca/Titan.git"
  },
  "scripts": {},
  "devDependencies": {
    "@nrwl/cli": "15.9.3",
    "lerna": "6.6.1",
    "nx": "16.10.0"
  },
  "engines": {
    "node": "^20.4.0 || ^18.17.1"
  }
}

[dep]

It's probably worth mentioning that our tech stack isn't in a place right now where we can upgrade to Lerna 7.x, due to this reason:

legacy package management commands have been removed We no longer include the bootstrap, add, and link commands by default. We strongly recommend using your package manager (npm, yarn, pnpm) for package management related concerns such as installing and linking dependencies.

We have a pretty big monorepo that's deeply embedded into lerna's bootstrapping command. it's a complicated setup with a lot of hoisting for varying dependency versions, and long story short, we want to keep things as they are with Lerna 6 until we have a critically burning need to upgrade (as it's working great right now for our needs)

[JamesHenry]

@dep Ah right yep so the solution here would be to go ahead and migrate to lerna v7 but install the @lerna/legacy-package-management explicitly (at the same v7 version) in your package.json

For vanilla lerna users that don't also use on nx, staying on v6 isn't an issue. But if you use nx alongside lerna, you can't stay back on old lerna versions because you'll be bringing in mixed majors of nx.

FWIW I have found that yarn 3+ is the best substitute for lerna bootstrap for complex hoisting/no hoisting setups, because it offers you a ton of control of how and when things get hoisted. You can disable it entirely for the monorepo, or control it on a package by package basis, as well as for certain dependencies. The other package managers don't offer you as much fine grained control of hoisting I don't think.

[JamesHenry]

P.S. in your example

"@nrwl/cli": "15.9.3", "nx": "16.10.0"

You have two different majors of nx explicitly there - that is never correct. You should never have nx + any @nx/ or @nrwl/ packages be on different versions from each other. They should always be identical. (In modern nx @nrwl/cli won't be there at all, but speaking generally)

[dep]

@JamesHenry Thank you for the info and help!

[JamesHenry]

Happy to help!

[github-actions[bot]]

This issue has been closed for more than 30 days. If this issue is still occuring, please open a new issue with more recent context.