search

npm / cli

the package manager for JavaScript
https://docs.npmjs.com/cli/
Other
8.44k stars 3.14k forks source link

[BUG] Failes to compile to a Single executable application #7845

Closed guest271314 closed 5 days ago

guest271314 commented 6 days ago

Is there an existing issue for this?

This issue exists in the latest npm version

Current Behavior

node v23.0.0-nightly20241016019efe1453 fails to compile ../lib/node_modules/npm/bin/npm-cli.js (symlinked from /node-v23/bin/npm in extracted Node.js nightly archive) to a single executable application. https://nodejs.org/api/single-executable-applications.html

user@user:~/bin$ ./node_modules/postject/dist/cli.js npm  NODE_SEA_BLOB sea-prep.blob --sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2 --overwrite
Start injection of NODE_SEA_BLOB in npm...
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note'
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note.100'
šŸ’‰ Injection done!
user@user:~/bin$ npm --help
(node:56496) Warning: Currently the require() provided to the main script embedded into single-executable applications only supports loading built-in modules.
To load a module from disk after the single executable application is launched, use require("module").createRequire().
Support for bundled module loading or virtual file systems are under discussions in https://github.com/nodejs/single-executable
(Use `npm --trace-warnings ...` to show where the warning was created)
node:internal/main/embedding:111
    throw new ERR_UNKNOWN_BUILTIN_MODULE(id);
    ^

Error [ERR_UNKNOWN_BUILTIN_MODULE]: No such built-in module: ../lib/cli.js
    at embedderRequire (node:internal/main/embedding:111:11)
    at node-v23/bin/npm:2:1
    at embedderRunCjs (node:internal/main/embedding:87:10) {
  code: 'ERR_UNKNOWN_BUILTIN_MODULE'
}

Node.js v23.0.0-nightly20241016019efe1453

It looks like the issue is related to require()ing files on disk at

//#!/usr/bin/env node
require('../lib/cli.js')(process)

instead of modules. See https://nodejs.org/api/single-executable-applications.html#requireid-in-the-injected-main-script-is-not-file-based

require() in the injected main script is not the same as the require() available to modules that are not injected. It also does not have any of the properties that non-injected require() has except require.main. It can only be used to load built-in modules. Attempting to load a module that can only be found in the file system will throw an error.

I tried bundling with bun build first, though that also fails - because bun build only emits Ecmascript Modules, and Node.js' Single executable application implementation only supports CommonJS.

Expected Behavior

node to be capable of compiling npm CLI source code to a single executable application.

Bun works out of the box

bun build ~/bin/node-v23/bin/npm --compile --outfile=bun-npm
bun-npm --help
npm <command>

Usage:

npm install        install all the dependencies in your project
npm install <foo>  add the <foo> dependency to your project
npm test           run this project's tests
npm run <foo>      run the script named <foo>
npm <command> -h   quick help on <command>
npm -l             display usage info for all commands
npm help <term>    search for help on <term>
npm help npm       more involved overview

All commands:

    access, adduser, audit, bugs, cache, ci, completion,
    config, dedupe, deprecate, diff, dist-tag, docs, doctor,
    edit, exec, explain, explore, find-dupes, fund, get, help,
    help-search, hook, init, install, install-ci-test,
    install-test, link, ll, login, logout, ls, org, outdated,
    owner, pack, ping, pkg, prefix, profile, prune, publish,
    query, rebuild, repo, restart, root, run-script, sbom,
    search, set, shrinkwrap, star, stars, start, stop, team,
    test, token, uninstall, unpublish, unstar, update, version,
    view, whoami

Specify configs in the ini-formatted file:
    /home/user/.npmrc
or on the command line via: npm <command> --key=value

More configuration info: npm help config
Configuration fields: npm help 7 config

npm@10.9.0 /home/user/bin/node-v23/lib/node_modules/npm

Steps To Reproduce

  1. Fetch and extract Node.js nightly archive to folder named node-v23
  2. Move node to PATH in ~/bin
  3. Fetch postject: bun install postject
  4. Comment shebang line #!/usr/bin/env node in ../lib/node_modules/npm/bin/npm-cli.js because we have not installed node globally
  5. Follow instructions here https://nodejs.org/api/single-executable-applications.html#single-executable-applications in ~/bin directory 
    
cp node npm

echo '{ "main": "node-v23/bin/npm", "output": "sea-prep.blob" }' > sea-config.json

./node_modules/postject/dist/cli.js npm NODE_SEA_BLOB sea-prep.blob --sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2 --overwrite


### Environment

- npm: 10.9.0
- Node.js: v23.0.0-nightly20241016019efe1453
- OS Name: Linux
- System Model Name:
- npm config:
```ini
; node bin location = /home/user/bin/bun
; node version = v22.6.0
; npm local prefix = /home/user/bin
; npm version = 10.9.0
; cwd = /home/user/bin
; HOME = /home/user
milaninfy commented 5 days ago

This doesn't seem like issue with npm. I would suggest to file this issue at relevant project.

guest271314 commented 5 days ago

I figured it out.

npm i npm # Using npm built with deno compile
bun build /home/user/bin/node_modules/npm/bin/npm-cli.js --target=node --outfile=bun-npm-bundle.js

  bun-npm-bundle.js  9.20 KB

[17ms] bundle 4 modules
cp node node-npm

Modify the Ecmascript Module output by bun build to be a CommonJS module

bun-npm-bundle.js

In pertinent part

#!/usr/bin/env node
const { createRequire } = require('node:module');
require = createRequire(__filename); 
var __commonJS = (cb, mod) => () => (mod || cb((mod = { exports: {} }).exports, mod), mod.exports);
// var __require = /* @__PURE__ */ createRequire(import.meta.url);

// ...

// node_modules/npm/lib/cli.js
var require_cli = __commonJS((exports, module) => {
  var __dirname = "/home/user/bin/node_modules/npm/lib";
  var validateEngines = require_validate_engines();
  var cliEntry = require("node:path").resolve(__dirname, "cli/entry.js");
  module.exports = (process2) => validateEngines(process2, () => require(cliEntry));
});

// node_modules/npm/bin/npm-cli.js
var require_npm_cli = __commonJS(() => {
  require_cli()(process);
});

module.exports = require_npm_cli();
// export default require_npm_cli();
node --experimental-sea-config sea-config.json 
Wrote single executable preparation blob to sea-prep.blob
./node_modules/postject/dist/cli.js node-npm  NODE_SEA_BLOB sea-prep.blob --sentinel-fuse NODE_SEA_FUSE_fce680ab2cc467b6e072b8b5df1996b2 --overwrite
Start injection of NODE_SEA_BLOB in node-npm...
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note'
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note.100'
warning: Can't find string offset for section name '.note.100'
šŸ’‰ Injection done!
node-npm
(node:66302) ExperimentalWarning: Single executable application is an experimental feature and might change at any time
(Use `node-npm --trace-warnings ...` to show where the warning was created)
npm warn cli npm v10.9.0 does not support Node.js v23.0.0-nightly20241016019efe1453. This version of npm supports the following node versions: `^18.17.0 || >=20.5.0`. You can find the latest version at https://nodejs.org/.
npm <command>

Usage:

npm install        install all the dependencies in your project
npm install <foo>  add the <foo> dependency to your project
npm test           run this project's tests
npm run <foo>      run the script named <foo>
npm <command> -h   quick help on <command>
npm -l             display usage info for all commands
npm help <term>    search for help on <term>
npm help npm       more involved overview

All commands:

    access, adduser, audit, bugs, cache, ci, completion,
    config, dedupe, deprecate, diff, dist-tag, docs, doctor,
    edit, exec, explain, explore, find-dupes, fund, get, help,
    help-search, hook, init, install, install-ci-test,
    install-test, link, ll, login, logout, ls, org, outdated,
    owner, pack, ping, pkg, prefix, profile, prune, publish,
    query, rebuild, repo, restart, root, run-script, sbom,
    search, set, shrinkwrap, star, stars, start, stop, team,
    test, token, uninstall, unpublish, unstar, update, version,
    view, whoami

Specify configs in the ini-formatted file:
    /home/user/.npmrc
or on the command line via: npm <command> --key=value

More configuration info: npm help config
Configuration fields: npm help 7 config

npm@10.9.0 /home/user/bin/node_modules/npm