isbinaryfile dependency update breaking yarn add teraslice-cli in assets CI workflow

[busma13]

isbinaryfile updated node engine to >20.0.0 in version 5.0.1. They then downgrade node version to >18.0.0 in version 5.0.2. Version 5.0.0 is the last version that supports node 16.

• In teraslice-cli, isBinaryFile is required by yeoman-environment (version ^4.0.10) and also by its dependency mem-fs-editor (version ^5.0.0).
• teraslice-cli is used by the assets CI to build the assets (yarn global add teraslice-cli)
• failed CI example: https://github.com/terascope/standard-assets/actions/runs/7907928098/job/21632825647
• We have tried adding "isbinaryfile": "5.0.0" to the resolutions in the root directory as well as in teraslice-cli. This resolves the error when installing dependencies in the monorepo, but we still get a node engine error when installing the package using yarn global.

[godber]

It's worth pointing out that we're just about ready to cut away from Node 16, we have on internal project that hasn't shipped all the way to prod on the Node 18 build, but does work and has been working in test environments. Hypothetically we could drop 16 support and move on to adding 20 and this problem would go away. The risk being that we find some reason we can't get that internal project shipped on 18 and need some test coverage on Node 16 still.

[godber]

The upstream package rolled back to node 16 here: https://github.com/gjtorikian/isBinaryFile/issues/70

If he rolled back to 16 our problem would go away, but I almost feel bad asking it.

We've tried about 4 different ways of trying to constrain this dependency to and older version without any luck.

[godber]

The only additional progress I've made on this is that it's pretty clear that yarn only supports resolutions at the top level package.json in the case of workspaces/monorepos.

[busma13]

Pulling teraslice-cli files out of the monorepo and into their own project, then running yarn install results in the correct versions of isbinaryfile. I can also run yarn global add file:<path to standalone teraslice-cli> and I get the same yarn.lock file with the correct versions. It is only when the package is pulled directly from npm that the yarn.lock is built with an isbinaryfile with version 5.0.2.

[godber]

Deleting yarn.lock on master and re-running yarn results in a similar engine error but for Elasticsearch:

error elasticsearch8@8.12.1: The engine "node" is incompatible with this module. Expected version ">=18". Got "16.20.2"
verbose 27.735799503 Error: Found incompatible module.
    at checkOne (/Users/godber/Workspace/terascope/opensource/teraslice/.yarn/releases/yarn-1.22.19.js:48581:11)
    at Object.check (/Users/godber/Workspace/terascope/opensource/teraslice/.yarn/releases/yarn-1.22.19.js:48600:5)
    at /Users/godber/Workspace/terascope/opensource/teraslice/.yarn/releases/yarn-1.22.19.js:6894:73
    at Generator.next (<anonymous>)
    at step (/Users/godber/Workspace/terascope/opensource/teraslice/.yarn/releases/yarn-1.22.19.js:310:30)
    at /Users/godber/Workspace/terascope/opensource/teraslice/.yarn/releases/yarn-1.22.19.js:321:13
error Found incompatible module.

[godber]

At this point, I think I'll advocate for eliminating Node 16 from our support matrix and hope we're not forced to do any more releases on it. I will spend a few minutes seeing what all uses the yeoman dependencies and if they actually can be tossed out. Though I have a suspicion that it's used for more than just templating asset generation.

[godber]

I don't think we can just eliminate the yeoman-generator as a dependency. It is used by generator-teraslice which generates the asset registries in addition to generating sample assets. So it's not something we can trivially eliminate.

[busma13]

The reason our resolutions from the teraslice-cli package.json are not being used when we run yarn global add teraslice-cli is because that command is adding a dependency to the yarn global 'project' package.json. Because resolutions are only followed if in the root package.json of a project, this is where we must add resolutions.

For example:

• Run yarn global dir locally to get the path to yarn global: /Users/<myusername>/.config/yarn/global.
• Run ls and you will see node_modules, package.json, and yarn.lock.

package.json:

{
  "dependencies": {
    "jest": "^29.7.0",
    "node-gyp": "^9.4.0",
    "teraslice-cli": "^0.61.5",
    "typescript": "^5.3.3"
  }
}

If I add a resolutions section with the packages/versions I want to limit it will look like this:

{
  "dependencies": {
    "jest": "^29.7.0",
    "node-gyp": "^9.4.0",
    "teraslice-cli": "^0.61.5",
    "typescript": "^5.3.3"
  },
  "resolutions": {
    "isbinaryfile": "<5.0.1",
    "yeoman-environment/isbinaryfile": "4.0.10"
  }
}

Now running yarn global add teraslice-cli while on node 16 will install teraslice-cli without a node engine error.

This modification of the package.json could be done within a CI runner if we run into this type of issue in the future.

[busma13]

We updated the node version to bypass this issue. Closing