Open ealib opened 1 year ago
For comparison, this script
'use strict'
const md = require('node-mdaemon-api');
const moduleInfo = md.getModuleInfo();
console.dir(moduleInfo);
works correctly under Node v18.13.0.
Node.js refuses to load this file:
node:internal/errors:484
ErrorCaptureStackTrace(err);
^
TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".node" for /Users/ib/dev/test_napi/asdf/asdf.darwin-arm64.node
at new NodeError (node:internal/errors:393:5)
at Object.getFileProtocolModuleFormat [as file:] (node:internal/modules/esm/get_format:75:9)
at defaultGetFormat (node:internal/modules/esm/get_format:114:38)
at defaultLoad (node:internal/modules/esm/load:81:20)
at nextLoad (node:internal/modules/esm/loader:161:28)
at ESMLoader.load (node:internal/modules/esm/loader:594:26)
at ESMLoader.moduleProvider (node:internal/modules/esm/loader:446:22)
at new ModuleJob (node:internal/modules/esm/module_job:64:26)
at #createModuleJob (node:internal/modules/esm/loader:469:17)
at ESMLoader.getModuleJob (node:internal/modules/esm/loader:423:34) {
code: 'ERR_UNKNOWN_FILE_EXTENSION'
}
From what I gather native extensions must always be require()
d. We need to improve error message here.
Thank you @bartlomieju for taking the time to verify this behaviour. Unfortunately I forgot to point out that the native node-mdaemon-api module (I am the author) only exists for Windows x64, not for Mac.
I created a repository with these scripts by specifying exactly the requirements in the package.json
and, in particular, "type": "module"
, to force Node to operate in ESM mode, thus explicitly requiring native modules to be loaded with require
, as you observed.
Thank you @bartlomieju for taking the time to verify this behaviour. Unfortunately I forgot to point out that the native node-mdaemon-api module (I am the author) only exists for Windows x64, not for Mac.
No worries, I figured that out and created a simple project that does the same thing but can be compiled on ARM.
I created a repository with these scripts by specifying exactly the requirements in the
package.json
and, in particular,"type": "module"
, to force Node to operate in ESM mode, thus explicitly requiring native modules to be loaded withrequire
, as you observed.
Sure, I'll try to look into fixing this problem next week - we should at least have a nice error saying what's wrong here. Side note: it's surprising that there's still no way to import Node API modules via ESM.
With
C:\>deno --version
deno 1.33.3 (release, x86_64-pc-windows-msvc)
v8 11.4.183.2
typescript 5.0.4
the error message has changed
error: Unable to load C:\Users\UserName\AppData\Local\deno\npm\registry.npmjs.org\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api\node-mdaemon-api.node imported from file:///C:/tmp/test-native.ts
Caused by:
invalid utf-8 sequence of 1 bytes from index 2
As @loynoir suggested in #19130 , I tried to directly import
the native binary module, but Deno fails with the same error:
C:\Users\UserName\deno-issue-17246>deno repl -A
Deno 1.33.4
exit using ctrl+d, ctrl+c, or close()
> const md = await import('npm:node-mdaemon-api@23.0.0-alpha.21/node-mdaemon-api.node');
Uncaught TypeError: Unable to load C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api\node-mdaemon-api.node
Caused by:
invalid utf-8 sequence of 1 bytes from index 2
at async <anonymous>:2:13
>
@ealib
I did not suggest that.
I suggest
> await import('npm:node-mdaemon-api@23.0.0-alpha.21/cjs-wrapper-to-node-mdaemon-api-node.cjs');
@loynoir Sorry, I didn't realise you were suggesting writing a CJS module that in turn would take care of loading the native module.
I wrote a simple one index.cjs
'use strict'
const md = require('./node-mdaemon-api.node');
module.exports = md;
and running the minimal test.ts
script
deno run --allow-all test.ts
const md = await import ('npm:node-mdaemon-api@23.0.0-alpha.21/index.cjs');
now outputs
C:\Users\UserName\deno-issue-17246>deno run --allow-all test.ts
error: Uncaught TypeError: LoadLibraryExW failed
at Object.Module._extensions..node (ext:deno_node/01_require.js:1029:24)
at Module.load (ext:deno_node/01_require.js:880:32)
at Function.Module._load (ext:deno_node/01_require.js:714:12)
at Module.require (ext:deno_node/01_require.js:902:19)
at require (ext:deno_node/01_require.js:1042:16)
at Object.<anonymous> (file:///C:/Users/UserName/deno-issue-17246/node_modules/.deno/node-mdaemon-api@23.0.0-alpha.21/node_modules/node-mdaemon-api/index.cjs:2:12)
at Object.<anonymous> (file:///C:/Users/UserName/deno-issue-17246/node_modules/.deno/node-mdaemon-api@23.0.0-alpha.21/node_modules/node-mdaemon-api/index.cjs:4:4)
at Module._compile (ext:deno_node/01_require.js:969:34)
at Object.Module._extensions..js (ext:deno_node/01_require.js:1002:10)
at Module.load (ext:deno_node/01_require.js:880:32)
where I see that Deno finds the script correctly (it is in the same directory as node-mdaemon-api.node
and Node executes it correctly), but then Deno itself fails to open the DLL file (LoadLibraryExW
).
@ealib
Are you using Node-API
?
https://github.com/denoland/deno/issues/18444#issuecomment-1484151518
@loynoir yes, I am (via node-addon-api 6.1.0).
With current deno --version
deno 1.36.0 (release, x86_64-pc-windows-msvc)
v8 11.6.189.12
typescript 5.1.6
you get a slightly different message when running test-native.ts
:
C:\Users\UserName\deno-issue-17246>npm run start-deno
> deno-issue-17246@1.0.0 start-deno
> deno run --allow-all test-native.ts
error: Could not find npm package 'node-mdaemon-api' matching '*'.
at file:///C:/Users/UserName/deno-issue-17246/test-native.ts:1:31
C:\Users\UserName\deno-issue-17246>
However, when running the minimal script
const md = await import ('npm:node-mdaemon-api@23.0.0-alpha.21/index.cjs');
but having placed the index.cjs
file in the package, as suggested by @loynoir , the same LoadLibraryExW
issue appears as with previous version:
C:\Users\UserName\deno-issue-17246>deno run -A --log-level debug test.ts
DEBUG RS - deno::args::package_json:147 - package.json file found at 'C:\Users\UserName\deno-issue-17246\package.json'
DEBUG RS - deno::cache::cache_db:129 - Opening cache C:\Users\Utente\AppData\Local\deno\dep_analysis_cache_v1...
DEBUG RS - deno::cache::cache_db:129 - Opening cache C:\Users\Utente\AppData\Local\deno\node_analysis_cache_v1...
DEBUG RS - deno::js:10 - Deno isolate init with snapshots.
DEBUG JS - args []
DEBUG RS - deno::worker:140 - main_module file:///C:/Users/UserName/deno-issue-17246/test.ts
DEBUG RS - deno::module_loader:111 - Preparing module load.
DEBUG RS - deno::module_loader:125 - Creating module graph.
DEBUG RS - deno::file_fetcher:516 - FileFetcher::fetch() - specifier: file:///C:/Users/UserName/deno-issue-17246/test.ts
DEBUG RS - deno_runtime::permissions:86 - ⚠️️ Granted read access to "C:\Users\UserName\deno-issue-17246\test.ts"
DEBUG RS - deno_npm::resolution::snapshot:712 - Resolved node-mdaemon-api@23.0.0-alpha.21 to node-mdaemon-api@23.0.0-alpha.21
DEBUG RS - deno_npm::resolution::graph:972 - <package-req> - Resolved node-mdaemon-api@23.0.0-alpha.21 to node-mdaemon-api@23.0.0-alpha.21
DEBUG RS - deno::util::fs:546 - Acquiring file lock at C:\Users\UserName\deno-issue-17246\node_modules\.deno\.deno.lock
DEBUG RS - deno::util::fs:565 - Acquired file lock at C:\Users\UserName\deno-issue-17246\node_modules\.deno\.deno.lock
DEBUG RS - deno::module_loader:188 - Prepared module load.
DEBUG RS - deno_core::runtime::bindings:272 - dyn_import specifier npm:node-mdaemon-api@23.0.0-alpha.21/index.cjs referrer file:///C:/Users/UserName/deno-issue-17246/test.ts
DEBUG RS - deno::npm::resolvers:121 - Resolved package folder of node-mdaemon-api@23.0.0-alpha.21 to C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api
DEBUG RS - deno::npm::resolvers:121 - Resolved package folder of node-mdaemon-api@23.0.0-alpha.21 to C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api
DEBUG RS - deno::npm::resolvers:121 - Resolved package folder of node-mdaemon-api@23.0.0-alpha.21 to C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api
DEBUG RS - deno::npm::resolvers:141 - Resolved package folder of file:///C:/Users/UserName/deno-issue-17246/node_modules/.deno/node-mdaemon-api@23.0.0-alpha.21/node_modules/node-mdaemon-api/index.cjs to C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api
DEBUG RS - deno::npm::resolvers:121 - Resolved package folder of node-mdaemon-api@23.0.0-alpha.21 to C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api
DEBUG RS - deno::npm::resolvers:141 - Resolved package folder of file:///C:/Users/UserName/deno-issue-17246/node_modules/.deno/node-mdaemon-api@23.0.0-alpha.21/node_modules/node-mdaemon-api/index.cjs to C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api
DEBUG RS - deno::npm::resolvers:141 - Resolved package folder of file:///C:/Users/UserName/deno-issue-17246/node_modules/.deno/node-mdaemon-api@23.0.0-alpha.21/node_modules/node-mdaemon-api/index.cjs to C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api
DEBUG RS - deno::npm::resolvers:141 - Resolved package folder of file:///C:/Users/UserName/deno-issue-17246/node_modules/.deno/node-mdaemon-api@23.0.0-alpha.21/node_modules/node-mdaemon-api/index.cjs to C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api
DEBUG RS - deno::npm::resolvers:141 - Resolved package folder of file:///C:/Users/UserName/deno-issue-17246/node_modules/.deno/node-mdaemon-api@23.0.0-alpha.21/node_modules/node-mdaemon-api/index.cjs to C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api
DEBUG RS - deno::npm::resolvers:141 - Resolved package folder of file:///C:/Users/UserName/deno-issue-17246/node_modules/.deno/node-mdaemon-api@23.0.0-alpha.21/node_modules/node-mdaemon-api/index.cjs to C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api
DEBUG RS - deno_runtime::permissions:86 - ⚠️️ Granted ffi access to "C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api\node-mdaemon-api.node"
error: Uncaught TypeError: LoadLibraryExW failed
at Object.Module._extensions..node (node:module:778:26)
at Module.load (node:module:656:34)
at Function.Module._load (node:module:539:16)
at Module.require (node:module:675:23)
at require (node:module:789:20)
at Object.<anonymous> (file:///C:/Users/UserName/deno-issue-17246/node_modules/.deno/node-mdaemon-api@23.0.0-alpha.21/node_modules/node-mdaemon-api/index.cjs:2:12)
at Object.<anonymous> (file:///C:/Users/UserName/deno-issue-17246/node_modules/.deno/node-mdaemon-api@23.0.0-alpha.21/node_modules/node-mdaemon-api/index.cjs:6:4)
at Module._compile (node:module:731:36)
at Object.Module._extensions..js (node:module:745:12)
at Module.load (node:module:656:34)
C:\Users\UserName\deno-issue-17246>
It all seems to boil down to a problem in the LoadLibraryExW
call, as seen in these other issues: #16642 #17411
With current deno --version
deno 1.38.0 (release, x86_64-pc-windows-msvc)
v8 12.0.267.1
typescript 5.2.2
with same index.cjs
file in the package, running a test.ts
which contains the very bare code
const md = await import ('npm:node-mdaemon-api@23.0.0-alpha.21/index.cjs');
outputs
c:\Users\UserName\deno-issue-17246>deno run -A --log-level debug test.ts
DEBUG RS - deno::args::package_json:147 - package.json file found at 'C:\Users\UserName\deno-issue-17246\package.json'
DEBUG RS - deno::cache::cache_db:130 - Opening cache C:\Users\Utente\AppData\Local\deno\dep_analysis_cache_v1...
DEBUG RS - deno::cache::cache_db:130 - Opening cache C:\Users\Utente\AppData\Local\deno\node_analysis_cache_v1...
DEBUG RS - deno::js:11 - Deno isolate init with snapshots.
DEBUG JS - args []
DEBUG RS - deno::worker:150 - main_module file:///C:/Users/UserName/deno-issue-17246/test.ts
DEBUG RS - deno::module_loader:117 - Preparing module load.
DEBUG RS - deno::module_loader:132 - Creating module graph.
DEBUG RS - deno::file_fetcher:550 - FileFetcher::fetch() - specifier: file:///C:/Users/UserName/deno-issue-17246/test.ts
DEBUG RS - deno_runtime::permissions:86 - ⚠️️ Granted read access to "C:\Users\UserName\deno-issue-17246\test.ts"
DEBUG RS - deno_npm::resolution::snapshot:711 - Resolved node-mdaemon-api@23.0.0-alpha.21 to node-mdaemon-api@23.0.0-alpha.21
DEBUG RS - deno_npm::resolution::graph:972 - <package-req> - Resolved node-mdaemon-api@23.0.0-alpha.21 to node-mdaemon-api@23.0.0-alpha.21
DEBUG RS - deno::util::fs:577 - Acquiring file lock at C:\Users\UserName\deno-issue-17246\node_modules\.deno\.deno.lock
DEBUG RS - deno::util::fs:596 - Acquired file lock at C:\Users\UserName\deno-issue-17246\node_modules\.deno\.deno.lock
DEBUG RS - deno::module_loader:197 - Prepared module load.
DEBUG RS - deno_core::runtime::bindings:298 - dyn_import specifier npm:node-mdaemon-api@23.0.0-alpha.21/index.cjs referrer file:///C:/Users/UserName/deno-issue-17246/test.ts
DEBUG RS - deno::npm::managed:297 - Resolved package folder of node-mdaemon-api@23.0.0-alpha.21 to C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api
DEBUG RS - deno::npm::managed:297 - Resolved package folder of node-mdaemon-api@23.0.0-alpha.21 to C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api
DEBUG RS - deno::npm::managed:297 - Resolved package folder of node-mdaemon-api@23.0.0-alpha.21 to C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api
DEBUG RS - deno::npm::managed:297 - Resolved package folder of node-mdaemon-api@23.0.0-alpha.21 to C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api
DEBUG RS - deno::npm::managed:518 - Resolved package folder of file:///C:/Users/UserName/deno-issue-17246/node_modules/.deno/node-mdaemon-api@23.0.0-alpha.21/node_modules/node-mdaemon-api/index.cjs to C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api
DEBUG RS - deno::npm::managed:518 - Resolved package folder of file:///C:/Users/UserName/deno-issue-17246/node_modules/.deno/node-mdaemon-api@23.0.0-alpha.21/node_modules/node-mdaemon-api/index.cjs to C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api
DEBUG RS - deno_runtime::permissions:86 - ⚠️️ Granted ffi access to "C:\Users\UserName\deno-issue-17246\node_modules\.deno\node-mdaemon-api@23.0.0-alpha.21\node_modules\node-mdaemon-api\node-mdaemon-api.node"
error: Uncaught (in promise) TypeError: LoadLibraryExW failed
at Object.Module._extensions..node (node:module:780:24)
at Module.load (node:module:658:32)
at Function.Module._load (node:module:539:12)
at Module.require (node:module:677:19)
at require (node:module:791:16)
at Object.<anonymous> (file:///C:/Users/UserName/deno-issue-17246/node_modules/.deno/node-mdaemon-api@23.0.0-alpha.21/node_modules/node-mdaemon-api/index.cjs:2:12)
at Object.<anonymous> (file:///C:/Users/UserName/deno-issue-17246/node_modules/.deno/node-mdaemon-api@23.0.0-alpha.21/node_modules/node-mdaemon-api/index.cjs:4:4)
at Module._compile (node:module:733:34)
at Object.Module._extensions..js (node:module:747:10)
at Module.load (node:module:658:32)
c:\Users\UserName\deno-issue-17246>
See also #22652
Something changed recently, but it doesn't work yet.
It seems that Deno tries to load a binary *.node
file as a text file.
C:\deno-issue-17246>deno --version
deno 1.44.4 (release, x86_64-pc-windows-msvc)
v8 12.6.228.9
typescript 5.4.5
C:\deno-issue-17246>node --version
v20.15.0
C:\deno-issue-17246>yarn run start-node
yarn run v1.22.22
$ node test-native.js
{
fileName: '\\\\?\\C:\\deno-issue-17246\\node_modules\\.deno\\node-mdaemon-api@24.0.0-alpha.32\\node_modules\\node-mdaemon-api\\node-mdaemon-api.node',
isPrerelease: true,
isFreeVersion: true,
name: 'node-mdaemon-api',
version: {
full: '24.0.0-alpha.32',
major: 24,
minor: 0,
release: 0,
build: 173,
tag: 'alpha.32'
}
}
Done in 0.22s.
C:\deno-issue-17246>yarn run start-deno
yarn run v1.22.22
$ deno run --reload --allow-all test-native.ts
error: Unexpected character '�' at file:///C:/deno-issue-17246/node_modules/.deno/node-mdaemon-api@24.0.0-alpha.32/node_modules/node-mdaemon-api/node-mdaemon-api.node:1:3
MZ�♥♦���@►☺♫▼�♫� �!�☺L�!Th...
~
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
C:\deno-issue-17246>
I'm just checking whether it was left open by mistake:
C:\deno-issue-17246>deno --version
deno 2.0.0 (stable, release, x86_64-pc-windows-msvc)
v8 12.9.202.13-rusty
typescript 5.6.2
C:\deno-issue-17246>node --version
v20.18.0
C:\deno-issue-17246>yarn start
yarn run v1.22.22
$ node native.js
{
fileName: '\\\\?\\C:\\deno-issue-17246\\node_modules\\.deno\\node-mdaemon-api@24.0.2-alpha.34\\node_modules\\node-mdaemon-api\\node-mdaemon-api.node',
isPrerelease: true,
isFreeVersion: true,
name: 'node-mdaemon-api',
version: {
full: '24.0.2-alpha.34',
major: 24,
minor: 0,
release: 2,
build: 232,
tag: 'alpha.34'
}
}
Done in 0.35s.
C:\deno-issue-17246>deno task start:debug
Task start:debug deno run --reload --allow-all --log-level debug native.ts
DEBUG RS - deno_config::deno_json:779 - Config file found at 'C:\deno-issue-17246\deno.json'
DEBUG RS - deno_config::workspace::discovery:266 - package.json file found at 'C:\deno-issue-17246\package.json'
DEBUG RS - deno::args:930 - Finished config loading.
DEBUG RS - deno::cache::cache_db:168 - Opening cache C:\Users\User\AppData\Local\deno\dep_analysis_cache_v2...
DEBUG RS - deno::cache::cache_db:168 - Opening cache C:\Users\User\AppData\Local\deno\node_analysis_cache_v2...
DEBUG RS - deno::cache::cache_db:168 - Opening cache C:\Users\User\AppData\Local\deno\v8_code_cache_v2...
DEBUG RS - deno_config::workspace::resolver:315 - Workspace config generated this import map {
"imports": {
"@std/assert": "jsr:@std/assert@1",
"@std/assert/": "jsr:/@std/assert@1/"
}
}
DEBUG RS - deno::js:10 - Deno isolate init with snapshots.
DEBUG RS - deno::worker:182 - main_module file:///C:/deno-issue-17246/native.ts
DEBUG RS - import_map:1110 - Specifier "file:///C:/deno-issue-17246/native.ts" was not mapped in import map.
DEBUG RS - import_map:1110 - Specifier "file:///C:/deno-issue-17246/native.ts" was not mapped in import map.
DEBUG RS - deno::module_loader:111 - Preparing module load.
DEBUG RS - deno::module_loader:135 - Building module graph.
DEBUG RS - deno::file_fetcher:622 - FileFetcher::fetch_no_follow_with_options - specifier: file:///C:/deno-issue-17246/native.ts
DEBUG RS - import_map:1110 - Specifier "npm:node-mdaemon-api" was not mapped in import map.
DEBUG RS - deno::module_loader:181 - Prepared module load.
DEBUG RS - import_map:1110 - Specifier "file:///C:/deno-issue-17246/native.ts" was not mapped in import map.
error: Expected ';', '}' or <eof> at file:///C:/deno-issue-17246/node_modules/.deno/node-mdaemon-api@24.0.2-alpha.34/node_modules/node-mdaemon-api/node-mdaemon-api.node:1:3
MZ�♥♦���@►☺♫▼�♫� �!�☺L�!Th...
~
C:\deno-issue-17246>
Deno
Output
How to reproduce
Command
Script
test-native.ts
is