Dependency to deprecated node pregyp in node modules of hana-cli with issues

Hi there,

I am not able to build my mtar for my SAP CAP based application because of node-pre-gyp that is part of the hana-cli node modules. There are some dependencies with the deprecated node pregyp (latest is: @mapbox/node-pre-gyp": "^1.0.5"). I had to delete the hana-cli module and refresh my node modules in order to be able to successfully build an mtar file...because then the dependency to node pregyp was gone...

I already used these commands: npm uninstall node-pre-gyp --save npm install @mapbox/node-pre-gyp --save

In my package.json I have added the dependency: "@mapbox/node-pre-gyp": "^1.0.5",

...no effect... I am getting this error: ERROR could not package the "app-39818-srv" module when archiving: could not read the "/home/user/projects/app-39818-arbeitskleidung/gen/srv/node_modules/hana-cli/node_modules/.bin/detect-libc" symbolic link: stat /home/user/projects/app-39818-arbeitskleidung/gen/srv/node_modules/hana-cli/node_modules/detect-libc/bin/detect-libc.js: no such file or directory make: *** [Makefile_20210714125238.mta:37: app-39818-srv] Error 1 Error: could not build the MTA project: could not execute the "make -f Makefile_20210714125238.mta p=cf mtar= strict=true mode=" command: exit status 2

Please check this issue from your side as well.

Thank you and BR Rufat

First - I'm not sure why you are adding hana-cli to the dependencies of your CAP project. Generally you would just install hana-cli as a global package (npm install -g hana-cli). It really isn't necessary to install the tool within each project. And you definitely don't want to add it to your dependencies and install it in production along with your application. It would serve no purpose within your CAP application and would only balloon the node_modules folder unnecessarily.

As to the dependencies you mention - @mapbox/node-pre-gyp or node-pre-gyp - that isn't even a direct dependency of hana-cli. It must be coming from one of the inner modules - maybe the @sap/hana-client since it has platform dependent modules. But that error is talking about detect-libc which node-pre-gyp requires at ^1.0.2 - resolving to 1.0.3. But why that dependency isn't there - I have no idea.

But in the end I'm not sure how to recreate this error or even if it matters. As said earlier I don't see any reason to include this module in your MTAR (or the package.json dependencies in general). This module itself installs correctly on Windows, Mac and Linux; so I'm not sure what's different about what mbt is doing here. And even if it was some inner dependency (which I'm not sure it is), what's referenced is down in the stack of other modules not something I directly control, reference or use.

Finally are you building in BAS? If so are you sure you aren't just running out of file system space? The MBT build command takes a lot of space and its easy to hit the limit on the Dev Space File System during this command. That could cause the npm install within the MBT build step to fail and might explain why some dependencies are suddenly just missing.

Hi Thomas,

thank you for your detailed response! I was very busy last week, so I am responding now. So, yes, I am using BAS and yes it happened in the past that I ran out of file system space..actually I just installed hana-cli via "npm install -g hana-cli" as you already described...I am not putting any dependency manually in there...it just happened out of nowhere during the time I wrote to you...I will try again, after my development is safely deployed...right now, I can use the link to hana db explorer and I am fine....but I really just installed hana-cli and nothing else...it just seemed to me that it contains outdated package for node pregyp in the inner modules which produces this issue...but yeah I´ll have to check if it possible not to include it in my build process because as I said it did automatically create the dependencies..

BR Rufat

After trying the upgrade upgrade -g hana-cli I struggled with this issue: integrity checksum failed I uninstalled hana-cli and tried to install it again, but came now to the same issue as Rufat (think so): ERR! This is a bug in node-pre-gyp.


PS C:\Users\diraschk> npm install -g hana-cli  
C:\Program Files\nodejs\hana-cli -> C:\Program Files\nodejs\node_modules\hana-cli\bin\cli.js

> @sap/hana-client@2.9.23 install C:\Program Files\nodejs\node_modules\hana-cli\node_modules\@sap\hana-client
> node checkbuild.js

> sqlite3@5.0.2 install C:\Program Files\nodejs\node_modules\hana-cli\node_modules\sqlite3
> node-pre-gyp install --fallback-to-build

node-pre-gyp WARN Using request for node-pre-gyp https download 
node-pre-gyp ERR! UNCAUGHT EXCEPTION 
node-pre-gyp ERR! stack TypeError: tar.extract is not a function
node-pre-gyp ERR! stack     at Request.<anonymous> (C:\Users\diraschk\AppData\Roaming\nvm\v14.8.0\node_modules\hana-cli\node_modules\node-pre-gyp\lib\install.js:144:26)
node-pre-gyp ERR! stack     at Request.emit (events.js:326:22)
node-pre-gyp ERR! stack     at Request.onRequestResponse (C:\Users\diraschk\AppData\Roaming\nvm\v14.8.0\node_modules\hana-cli\node_modules\request\request.js:1059:10)
node-pre-gyp ERR! stack     at ClientRequest.emit (events.js:314:20)
node-pre-gyp ERR! stack     at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:640:27)
node-pre-gyp ERR! stack     at HTTPParser.parserOnHeadersComplete (_http_common.js:119:17)
node-pre-gyp ERR! stack     at TLSSocket.socketOnData (_http_client.js:509:22)
node-pre-gyp ERR! stack     at TLSSocket.emit (events.js:314:20)
node-pre-gyp ERR! stack     at addChunk (_stream_readable.js:303:12)
node-pre-gyp ERR! stack     at readableAddChunk (_stream_readable.js:279:9)
node-pre-gyp ERR! System Windows_NT 10.0.18363
node-pre-gyp ERR! command "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\hana-cli\\node_modules\\node-pre-gyp\\bin\\node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd C:\Program Files\nodejs\node_modules\hana-cli\node_modules\sqlite3
node-pre-gyp ERR! node -v v14.8.0
node-pre-gyp ERR! node-pre-gyp -v v0.11.0
node-pre-gyp ERR! This is a bug in `node-pre-gyp`.
node-pre-gyp ERR! Try to update node-pre-gyp and file an issue if it does not help:
node-pre-gyp ERR!     <https://github.com/mapbox/node-pre-gyp/issues>

> mbt@1.1.1 install C:\Program Files\nodejs\node_modules\hana-cli\node_modules\mbt
> binwrap-install

ERR TypeError: tar.x is not a function
    at C:\Users\diraschk\AppData\Roaming\nvm\v14.8.0\node_modules\hana-cli\node_modules\binwrap\binstall.js:23:8
    at new Promise (<anonymous>)
    at untgz (C:\Users\diraschk\AppData\Roaming\nvm\v14.8.0\node_modules\hana-cli\node_modules\binwrap\binstall.js:21:10)
    at binstall (C:\Users\diraschk\AppData\Roaming\nvm\v14.8.0\node_modules\hana-cli\node_modules\binwrap\binstall.js:11:12)
    at install (C:\Users\diraschk\AppData\Roaming\nvm\v14.8.0\node_modules\hana-cli\node_modules\binwrap\install.js:20:10)
    at Object.install (C:\Users\diraschk\AppData\Roaming\nvm\v14.8.0\node_modules\hana-cli\node_modules\binwrap\index.js:14:14)
    at Object.<anonymous> (C:\Users\diraschk\AppData\Roaming\nvm\v14.8.0\node_modules\hana-cli\node_modules\binwrap\bin\binwrap-install:18:9)
    at Module._compile (internal/modules/cjs/loader.js:1251:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1272:10)
    at Module.load (internal/modules/cjs/loader.js:1100:32)
    at Function.Module._load (internal/modules/cjs/loader.js:962:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
    at internal/main/run_main_module.js:17:47

> @sap/cds-compiler@2.4.4 postinstall C:\Program Files\nodejs\node_modules\hana-cli\node_modules\@sap\cds-compiler
> node lib/fix_antlr4-8_warning.js

postinstall cds-compiler: Successfully fixed Antlr's circular dependency

> @sap-cloud-sdk/core@1.46.0 postinstall C:\Program Files\nodejs\node_modules\hana-cli\node_modules\@sap-cloud-sdk\core
> node usage-analytics.js

npm WARN optional SKIPPING OPTIONAL DEPENDENCY: sqlite3@5.0.2 (node_modules\hana-cli\node_modules\sqlite3):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: sqlite3@5.0.2 install: `node-pre-gyp install --fallback-to-build`
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Exit status 7
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: mbt@1.1.1 (node_modules\hana-cli\node_modules\mbt):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: mbt@1.1.1 install: `binwrap-install`
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Exit status 1

+ hana-cli@2.202107.3
added 1149 packages from 725 contributors in 237.129s

checking the version leads to this error message: Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: C:\Users\diraschk\AppData\Roaming\nvm\v14.8.0\node_modules\hana-cli\node_modules\latest-version\index.js


PS C:\Users\diraschk> hana-cli version
hana-cli: 2.202107.3
@sap/audit-logging: 5.0.0
@sap/cds: 5.3.2
@sap/cds-compiler: 2.4.4
@sap/cds-foss: 2.3.1
@sap/cds-dk: 4.3.3
@sap/cds-sidecar-client: 1.1.21
@sap/eslint-plugin-cds: 2.0.4
@sap/hana-client: 2.9.23
@sap/hdbext: 7.3.0
@sap/e2e-trace: 3.0.0
@sap/textbundle: 4.0.0
@sap/xsenv: 3.1.1
@sap/xssec: 3.2.2
sap-hdbext-promisfied: 2.202107.1
hana-cli home: C:\Users\diraschk\AppData\Roaming\nvm\v14.8.0\node_modules\hana-cli\bin
Node.js: v14.8.0
Change Log: https://github.com/SAP-samples/hana-developer-cli-tool-example/blob/main/CHANGELOG.md
Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: C:\Users\diraschk\AppData\Roaming\nvm\v14.8.0\node_modules\hana-cli\node_modules\latest-version\index.js
require() of ES modules is not supported.
require() of C:\Users\diraschk\AppData\Roaming\nvm\v14.8.0\node_modules\hana-cli\node_modules\latest-version\index.js from C:\Users\diraschk\AppData\Roaming\nvm\v14.8.0\node_modules\hana-cli\bin\version.js is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from C:\Users\diraschk\AppData\Roaming\nvm\v14.8.0\node_modules\hana-cli\node_modules\latest-version\package.json.

WSL installation worked fine for me. Get only the same error msg while running command hana-cli version Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: --> don't know what this means for me. do I've to do anything?


diraschk@vmn-it-ful-05:/mnt/c/git/priv$ hana-cli version
hana-cli: 2.202107.3
@sap/audit-logging: 5.0.0
@sap/cds: 5.3.2
@sap/cds-compiler: 2.4.4
@sap/cds-foss: 2.3.1
@sap/cds-dk: 4.3.3
@sap/cds-sidecar-client: 1.1.21
@sap/eslint-plugin-cds: 2.0.4
@sap/hana-client: 2.9.23
@sap/hdbext: 7.3.0
@sap/e2e-trace: 3.0.0
@sap/textbundle: 4.0.0
@sap/xsenv: 3.1.1
@sap/xssec: 3.2.2
sap-hdbext-promisfied: 2.202107.1
hana-cli home: /mnt/c/Users/diraschk/AppData/Roaming/nvm/v14.8.0/node_modules/hana-cli/bin
Node.js: v14.17.3
Change Log: https://github.com/SAP-samples/hana-developer-cli-tool-example/blob/main/CHANGELOG.md
Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /mnt/c/Users/diraschk/AppData/Roaming/nvm/v14.8.0/node_modules/hana-cli/node_modules/latest-version/index.js
require() of ES modules is not supported.
require() of /mnt/c/Users/diraschk/AppData/Roaming/nvm/v14.8.0/node_modules/hana-cli/node_modules/latest-version/index.js from /mnt/c/Users/diraschk/AppData/Roaming/nvm/v14.8.0/node_modules/hana-cli/bin/version.js is an ES module 
file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from 
/mnt/c/Users/diraschk/AppData/Roaming/nvm/v14.8.0/node_modules/hana-cli/node_modules/latest-version/package.json.

The version command error is my fault. I updated a node module I was using within that command it had an incompatible change. I've fixed that and will push to npm later today.

As far as the other errors, they are coming from the sqlite3 module which in turn is coming in from @sap/cds-dk. It's nothing that I'm using directly. Unfortunately sqlite3 has lots of open issues on these same. The @mapbox/node-pre-gyp for example: https://github.com/mapbox/node-sqlite3/issues/1479 and the other build issue: https://github.com/mapbox/node-sqlite3/issues/1472

But these errors would seem be safe to ignore. No functionality in hana-cli actually uses Sqlite. I tested an npm install, got these same errors but the tool still installed and worked fine.

Perhaps because I'm doing a npm dedupe to reduce the size of hana-cli overall there is some mixed dependencies when installing Sqlite within another tool. But again I'm only getting Sqlite here because I have to pull in @sap/cds-dk to get the compile.to.openAPI command (which is only in @sap/cds-dk not @sap/cds unfortunately).

I'll experiment with the npm dedupe and npm shrinkwrap process later to day to see if there is anything I can do about this. But for now it doesn't seem like it is actually causing any problems in the tool itself.

I have pushed the update to the version command and it should work fine now again. I did a few changes to the npm install before I prepare the shrinkwrap. On my machine at least it removed the nasty sqlite3 messages:

Would be interested to see if it improved things on your side as well.

Checked it for Windows Powershell and WSL: The installation worked fine in both cases and the ERR msg from version command (Error [ERR_REQUIRE_ESM]: Must use import to load ES Module) is gone.

Powershell

PS C:\Users\diraschk> npm upgrade -g hana-cli
npm WARN read-shrinkwrap This version of npm is compatible with lockfileVersion@1, but npm-shrinkwrap.json was generated for lockfileVersion@2. I'll try to do my best with it!
C:\Program Files\nodejs\hana-cli -> C:\Program Files\nodejs\node_modules\hana-cli\bin\cli.js

> @sap/hana-client@2.9.23 install C:\Program Files\nodejs\node_modules\hana-cli\node_modules\@sap\hdbext\node_modules\@sap\hana-client
> node checkbuild.js

> @sap/hana-client@2.9.23 install C:\Program Files\nodejs\node_modules\hana-cli\node_modules\sap-hdbext-promisfied\node_modules\@sap\hana-client
> node checkbuild.js

> mbt@1.1.1 install C:\Program Files\nodejs\node_modules\hana-cli\node_modules\@sap\cds-dk\node_modules\mbt
> binwrap-install

> sqlite3@5.0.2 install C:\Program Files\nodejs\node_modules\hana-cli\node_modules\@sap\cds-dk\node_modules\sqlite3 
> node-pre-gyp install --fallback-to-build

node-pre-gyp WARN Using request for node-pre-gyp https download 
[sqlite3] Success: "C:\Program Files\nodejs\node_modules\hana-cli\node_modules\@sap\cds-dk\node_modules\sqlite3\lib\binding\napi-v3-win32-x64\node_sqlite3.node" is installed via remote

> mbt@1.1.1 install C:\Program Files\nodejs\node_modules\hana-cli\node_modules\mbt
> binwrap-install

> @sap-cloud-sdk/core@1.47.1 postinstall C:\Program Files\nodejs\node_modules\hana-cli\node_modules\@sap-cloud-sdk\core
> node usage-analytics.js

> @sap/cds-compiler@2.4.4 postinstall C:\Program Files\nodejs\node_modules\hana-cli\node_modules\@sap\cds-dk\node_modules\@sap\cds-compiler
> node lib/fix_antlr4-8_warning.js

postinstall cds-compiler: Successfully fixed Antlr's circular dependency

> @sap-cloud-sdk/core@1.47.1 postinstall C:\Program Files\nodejs\node_modules\hana-cli\node_modules\@sap\cds-dk\node_modules\@sap-cloud-sdk\core
> node usage-analytics.js

npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.13 (node_modules\hana-cli\node_modules\@sap\cds-dk\node_modules\fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.13: wanted {"os":"darwin","arch":"any"} (current: {"os":"win32","arch":"x64"})

+ hana-cli@2.202107.4
added 1078 packages from 640 contributors, removed 58 packages and updated 48 packages in 302.629s
PS C:\Users\diraschk> hana-cli version       
hana-cli: 2.202107.4
@sap/audit-logging: 5.0.0
@sap/cds: 5.3.2
@sap/cds-compiler: 2.4.4
@sap/cds-foss: 2.3.1
@sap/cds-dk: 4.3.3
@sap/cds-sidecar-client: 1.1.21
@sap/eslint-plugin-cds: 2.0.4
@sap/hana-client: 2.9.23
@sap/hdbext: 7.3.0
@sap/e2e-trace: 3.0.0
@sap/textbundle: 4.0.0
@sap/xsenv: 3.1.1
@sap/xssec: 3.2.2
sap-hdbext-promisfied: 2.202107.1
hana-cli home: C:\Users\diraschk\AppData\Roaming\nvm\v14.8.0\node_modules\hana-cli\bin
Node.js: v14.8.0
Change Log: https://github.com/SAP-samples/hana-developer-cli-tool-example/blob/main/CHANGELOG.md
Latest hana-cli version available on npmjs.com: 2.202107.4

WSL

diraschk@vmn-it-ful-05:/mnt/c/git/priv$ hana-cli upgrade
diraschk@vmn-it-ful-05:/mnt/c/git/priv$ hana-cli version
hana-cli: 2.202107.4
@sap/audit-logging: 5.0.0
@sap/cds: 5.3.2
@sap/cds-compiler: 2.4.4
@sap/cds-foss: 2.3.1
@sap/cds-dk: 4.3.3
@sap/cds-sidecar-client: 1.1.21
@sap/eslint-plugin-cds: 2.0.4
@sap/hana-client: 2.9.23
@sap/hdbext: 7.3.0
@sap/e2e-trace: 3.0.0
@sap/textbundle: 4.0.0
@sap/xsenv: 3.1.1
@sap/xssec: 3.2.2
sap-hdbext-promisfied: 2.202107.1
hana-cli home: /mnt/c/Users/diraschk/AppData/Roaming/nvm/v14.8.0/node_modules/hana-cli/bin
Node.js: v14.17.3
Change Log: https://github.com/SAP-samples/hana-developer-cli-tool-example/blob/main/CHANGELOG.md
Latest hana-cli version available on npmjs.com: 2.202107.4

Thanks a lot!!!

SAP-samples / hana-developer-cli-tool-example

Dependency to deprecated node pregyp in node modules of hana-cli with issues #58