error in npm-config-validator

[wyardley]

What Renovate type, platform and version are you using? maybe more relevant for this one:

% node --version
v10.21.0
% npm --version
6.14.4
% npx --version
10.2.2

Describe the bug Using npx to run renovate-config-validator on a project's renovate config (which I think is a valid use case -- let me know if not), gives me the following error:

% npx renovate-config-validator     
 WARN: RE2 not usable, falling back to RegExp
       "err": {
         "code": "MODULE_NOT_FOUND",
         "message": "Cannot find module './build/Release/re2'",
         "stack": "Error: Cannot find module './build/Release/re2'\n    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)\n    at Function.Module._load (internal/modules/cjs/loader.js:562:25)\n    at Module.require (internal/modules/cjs/loader.js:692:17)\n    at require (internal/modules/cjs/helpers.js:25:18)\n    at Object.<anonymous> (/usr/local/lib/node_modules/renovate/node_modules/re2/re2.js:3:13)\n    at Module._compile (internal/modules/cjs/loader.js:778:30)\n    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)\n    at Module.load (internal/modules/cjs/loader.js:653:32)\n    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)\n    at Function.Module._load (internal/modules/cjs/loader.js:585:3)\n    at Module.require (internal/modules/cjs/loader.js:692:17)\n    at require (internal/modules/cjs/helpers.js:25:18)\n    at Object.<anonymous> (/usr/local/lib/node_modules/renovate/dist/util/regex.js:9:17)\n    at Module._compile (internal/modules/cjs/loader.js:778:30)\n    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)\n    at Module.load (internal/modules/cjs/loader.js:653:32)\n    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)\n    at Function.Module._load (internal/modules/cjs/loader.js:585:3)\n    at Module.require (internal/modules/cjs/loader.js:692:17)\n    at require (internal/modules/cjs/helpers.js:25:18)"
       }
Validating renovate.json
Validating config.js
OK

Relevant debug logs

To Reproduce run npx renovate-config-validator on a project with a valid renovate.json

Additional context

[viceice]

You have multiple issues:

• node v10 is no longer supported since renovae v20.0.0
• the re issue is a warning, not an error and can be ignored
• re possibly doesn't work on unsupported node v10

[wyardley]

It's a warning, but in this case, it's a warning that makes the rest of the output of the command very difficult to parse.

That said, I tested with node 12.19, and the result is even uglier:

% npx renovate-config-validator
 WARN: RE2 not usable, falling back to RegExp
       "err": {
         "code": "MODULE_NOT_FOUND",
         "requireStack": [
           "/usr/local/lib/node_modules/renovate/node_modules/re2/re2.js",
           "/usr/local/lib/node_modules/renovate/dist/util/regex.js",
           "/usr/local/lib/node_modules/renovate/dist/datasource/go/index.js",
           "/usr/local/lib/node_modules/renovate/dist/manager/bazel/extract.js",
           "/usr/local/lib/node_modules/renovate/dist/manager/bazel/index.js",
           "/usr/local/lib/node_modules/renovate/dist/manager/api.generated.js",
           "/usr/local/lib/node_modules/renovate/dist/manager/index.js",
           "/usr/local/lib/node_modules/renovate/dist/config/definitions.js",
           "/usr/local/lib/node_modules/renovate/dist/config/migration.js",
           "/usr/local/lib/node_modules/renovate/dist/config/file.js",
           "/usr/local/lib/node_modules/renovate/dist/config-validator.js"
         ],
         "message": "Cannot find module './build/Release/re2'\nRequire stack:\n- /usr/local/lib/node_modules/renovate/node_modules/re2/re2.js\n- /usr/local/lib/node_modules/renovate/dist/util/regex.js\n- /usr/local/lib/node_modules/renovate/dist/datasource/go/index.js\n- /usr/local/lib/node_modules/renovate/dist/manager/bazel/extract.js\n- /usr/local/lib/node_modules/renovate/dist/manager/bazel/index.js\n- /usr/local/lib/node_modules/renovate/dist/manager/api.generated.js\n- /usr/local/lib/node_modules/renovate/dist/manager/index.js\n- /usr/local/lib/node_modules/renovate/dist/config/definitions.js\n- /usr/local/lib/node_modules/renovate/dist/config/migration.js\n- /usr/local/lib/node_modules/renovate/dist/config/file.js\n- /usr/local/lib/node_modules/renovate/dist/config-validator.js",
         "stack": "Error: Cannot find module './build/Release/re2'\nRequire stack:\n- /usr/local/lib/node_modules/renovate/node_modules/re2/re2.js\n- /usr/local/lib/node_modules/renovate/dist/util/regex.js\n- /usr/local/lib/node_modules/renovate/dist/datasource/go/index.js\n- /usr/local/lib/node_modules/renovate/dist/manager/bazel/extract.js\n- /usr/local/lib/node_modules/renovate/dist/manager/bazel/index.js\n- /usr/local/lib/node_modules/renovate/dist/manager/api.generated.js\n- /usr/local/lib/node_modules/renovate/dist/manager/index.js\n- /usr/local/lib/node_modules/renovate/dist/config/definitions.js\n- /usr/local/lib/node_modules/renovate/dist/config/migration.js\n- /usr/local/lib/node_modules/renovate/dist/config/file.js\n- /usr/local/lib/node_modules/renovate/dist/config-validator.js\n    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:831:15)\n    at Function.Module._load (internal/modules/cjs/loader.js:687:27)\n    at Module.require (internal/modules/cjs/loader.js:903:19)\n    at require (internal/modules/cjs/helpers.js:74:18)\n    at Object.<anonymous> (/usr/local/lib/node_modules/renovate/node_modules/re2/re2.js:3:13)\n    at Module._compile (internal/modules/cjs/loader.js:1015:30)\n    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1035:10)\n    at Module.load (internal/modules/cjs/loader.js:879:32)\n    at Function.Module._load (internal/modules/cjs/loader.js:724:14)\n    at Module.require (internal/modules/cjs/loader.js:903:19)\n    at require (internal/modules/cjs/helpers.js:74:18)\n    at Object.<anonymous> (/usr/local/lib/node_modules/renovate/dist/util/regex.js:9:17)\n    at Module._compile (internal/modules/cjs/loader.js:1015:30)\n    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1035:10)\n    at Module.load (internal/modules/cjs/loader.js:879:32)\n    at Function.Module._load (internal/modules/cjs/loader.js:724:14)\n    at Module.require (internal/modules/cjs/loader.js:903:19)\n    at require (internal/modules/cjs/helpers.js:74:18)\n    at Object.<anonymous> (/usr/local/lib/node_modules/renovate/dist/datasource/go/index.js:25:17)\n    at Module._compile (internal/modules/cjs/loader.js:1015:30)"
       }
Validating renovate.json
Validating config.js
OK
%  node --version
v12.19.1

(note: did try doing an npm -g install re2, which did not seem to change anything here; also, if it's useful information, this is on Mac OS X)

[viceice]

you need to run npx -p re2 -p renovate renovate-config-validator to get rid of the warning, so npx will add optional re2 package to be usable by renovate

[wyardley]

That totally bombs:

% npx -p re2 -p renovate renovate-config-validator
npm ERR! cb.apply is not a function

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/wyardley/.npm/_logs/2020-11-19T16_45_38_236Z-debug.log
Install for [ 're2@latest', 'renovate@latest' ] failed with code 1

While I get that the error above is harmless, it is kind of messy, so I do wonder if there's a way to hide it.

[wyardley]

FWIW, run on our renovate config repo itself (with renovate as a devDependency and its regular dependencies installed), it works just fine in CI, including with node 10.x.

[rarkins]

The way any reasonable support policy works is that you can try to run whatever old version you want but if it's your desire that someone looks into it for you then you need to retest with a supported version first.

[wyardley]

The way any reasonable support policy works is that you can try to run whatever old version you want but if it's your desire that someone looks into it for you then you need to retest with a supported version first.

Agreed, which is why I tested it with node 12.

I wasn't previously aware of node 10 support being dropped, but doesn't seem to be related to this issue.

[rarkins]

Where/how is npx renovate-config-validator evaluated? I wasn't aware that npx can work with additional binaries in a package to begin with, and it seems to execute immediately (without install) locally:

❯ npx renovate-config-validator
Validating renovate.json
Validating config.js
OK

[wyardley]

I'm not an expert, but my understanding is that normally, npx will get what it needs, even if it's not installed in a given project.

So, typically, it's supposed to be able to get a program and exec it even if it's not currently installed. https://semantic-release.gitbook.io/semantic-release/support/faq#what-is-npx

In this case, I might often want to validate that a project's renovate config is valid, but I most likely would not want to have renovate as a devDependency for the project itself (or it might not even be a node project).

[rarkins]

But if somebody registered a malicious package called renovate-config-validator then would npx download and run that instead?

[wyardley]

But if somebody registered a malicious package called renovate-config-validator then would npx download and run that instead?

possibly, yes. This level of "magic" has always been a little concerning to me as well. I'm not enough of an expert to know if there's a way to "scope" a binary to make sure it's coming from a particular package, though I've never been able to find a way to do that -- I'm not actually sure how its logic works in terms of figuring out what to run / execute. if the dependency already exists either in the current project, or globally, it will run the binary from there. From a quick search, it looks like @viceice's example of using -p xyz is the way you're supposed to do that. I can test again.

I can confirm that if I globally install renovate first (which does bring in the re2 dependency), npx renovate-config-validator does work as expected.

% npx renovate-config-validator
Validating renovate.json
Validating config.js
OK

edit: with using -p renovate, though, I still get the error above:

% npx -p renovate renovate-config-validator
npm ERR! cb.apply is not a function

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/wyardley/.npm/_logs/2020-11-19T20_05_51_031Z-debug.log
Install for [ 'renovate@latest' ] failed with code 1

Potentially related to https://github.com/npm/npx/issues/67

[viceice]

Ok, looks like the npm issue to me. Try to clean your machine as suggested bin the npm issue.

[wyardley]

Yeah, either way, it works as expected with NPM globally installed, which I think is good enough. Thanks for the helpful responses.