paulmillr / noble-hashes

Audited & minimal JS implementation of hash functions, MACs and KDFs.
https://paulmillr.com/noble
MIT License
573 stars 46 forks source link

The new v0.4.2 seems to break imports from .mjs (Node.js ES module files) #17

Closed quentinadam closed 2 years ago

quentinadam commented 2 years ago

I am running the latest version of Node (v17.2.0) on Intel Mac OS X.

I have an ES Node.js package (with the "type":"module" flag), which is equivalent to having the code in an .mjs file.

Up until version 0.4.1, I used to do the following to use this package :

import { keccak_256 } from '@noble/hashes/lib/sha3';
console.log(keccak_256(new Uint8Array([1, 2, 3])));

With version 0.4.2, I am now getting the following error :

import { keccak_256 } from '@noble/hashes/lib/sha3';
         ^^^^^^^^^^
SyntaxError: Named export 'keccak_256' not found. The requested module '@noble/hashes/lib/sha3' is a CommonJS module, which may not support all module.exports as named exports.
CommonJS modules can always be imported via the default export, for example using:

import pkg from '@noble/hashes/lib/sha3';
const { keccak_256 } = pkg;

    at ModuleJob._instantiate (node:internal/modules/esm/module_job:127:21)
    at async ModuleJob.run (node:internal/modules/esm/module_job:191:5)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:331:24)
    at async loadESM (node:internal/process/esm_loader:88:5)
    at async handleMainPromise (node:internal/modules/run_main:65:12)

Node.js v17.2.0

I tried the suggestion :

import pkg from '@noble/hashes/lib/sha3';
const { keccak_256 } = pkg;
console.log(keccak_256(new Uint8Array([1, 2, 3])));

But am getting the following error :

(node:17801) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
(Use `node --trace-warnings ...` to show where the warning was created)
/Users/[redacted]/node_modules/@noble/hashes/lib/esm/sha3.js:1
import * as u64 from './_u64';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at Object.compileFunction (node:vm:352:18)
    at wrapSafe (node:internal/modules/cjs/loader:1026:15)
    at Module._compile (node:internal/modules/cjs/loader:1061:27)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1149:10)
    at Module.load (node:internal/modules/cjs/loader:975:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:190:29)
    at ModuleJob.run (node:internal/modules/esm/module_job:195:25)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:331:24)

Node.js v17.2.0

Changing my file extension to .cjs and using the old require() syntax works, but that somewhat defeats the purpose of trying to move to the new ES Module syntax.

This works (when running from .cjs file) :

const { keccak_256 } = require('@noble/hashes/lib/sha3');
console.log(keccak_256(new Uint8Array([1, 2, 3])));

Is there any possibilty to make this module work from an .mjs Node.js file ?

paulmillr commented 2 years ago

See https://github.com/paulmillr/noble-hashes/releases

Use lib/esm

quentinadam commented 2 years ago

Hello,

Thank you very much for the quick reply, and for pushing v0.4.3, but when changing the import to lib/esm, as follows:

import { keccak_256 } from '@noble/hashes/lib/esm/sha3';
console.log(keccak_256(new Uint8Array([1, 2, 3])));

I am getting the following issue :

node:internal/errors:464
    ErrorCaptureStackTrace(err);
    ^

Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/esm/sha3' is not defined by "exports" in /Users/[redacted]/node_modules/@noble/hashes/package.json imported from /Users/[redacted]/src/test.mjs
    at new NodeError (node:internal/errors:371:5)
    at throwExportsNotFound (node:internal/modules/esm/resolve:429:9)
    at packageExportsResolve (node:internal/modules/esm/resolve:683:3)
    at packageResolve (node:internal/modules/esm/resolve:853:14)
    at moduleResolve (node:internal/modules/esm/resolve:910:18)
    at defaultResolve (node:internal/modules/esm/resolve:1005:11)
    at ESMLoader.resolve (node:internal/modules/esm/loader:475:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:245:18)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:79:40)
    at link (node:internal/modules/esm/module_job:78:36) {
  code: 'ERR_PACKAGE_PATH_NOT_EXPORTED'
}

Node.js v17.2.0
paulmillr commented 2 years ago

@jacogr ping

jacogr commented 2 years ago

So you don't need to use @noble/hashes/lib/esm/sha3 just @noble/hashes/lib/sha3 - the reason it didn't work in 0.4.2 was because of the export map issue.

However, there is an issue where it doesn't find local imports, i.e. _u64 from sha3.js - not sure why that didn't show up for me (and doesn't show up in my project usage where I swapped to 0.4.3 this morning), however can replicate this on a brand new project. Will push a PR after I verified it works with a local build.

jacogr commented 2 years ago

Ok, so I would suggest reverting this (see https://github.com/paulmillr/noble-hashes/pull/18) until we can swap to tsc 4.6 which is not out yet - so 4.5 does add the correct relative import extensions in the outputs (assuming the module is set to node12), however from 4.5.2 this is marked unstable so there is some work remaining by the team.

Background reading here - https://github.com/microsoft/TypeScript/issues/42813

(Can add it manually via script, but rather not and let the compiler just compile correctly)

PS: Still confused as to why I don’t see this issue where I swapped to 0.4.3 this morning in my libs, but it was a certainly an issue in the esm output as it stands.

paulmillr commented 2 years ago

Why do you map to lib/sha3 instead of lib/sha3.js? Can't we just add an extension in pkg.json

jacogr commented 2 years ago

The issue is that with local imports for esm node always expects an extension -

will see if I can find some mapping solution

quentinadam commented 2 years ago

I believe you can put "./_u64.js" directly in the typescript code (the typescript compiler will ignore the .js extension and will link it up with the .ts file).

quentinadam commented 2 years ago

https://github.com/microsoft/TypeScript/issues/16577#issuecomment-309169829

jacogr commented 2 years ago

You are 100% right, that slipped my mind. Will adjust.

The only other small change then needed is a package.json in the lib/esm root containing only { “type”: “module” }

quentinadam commented 2 years ago

I see, that makes sense !