Unable to locate bindings or generate keys with Wasm backend. (node:4201)

[StringManolo]

My enviroment is Aarch64 Alpine Linux (musl)

Here is the warning:

(node:4577) Warning: Using WebAssembly backend: Could not locate the bindings file. Tried:
  → /projects/DarkMessengerCrypto/node_modules/pqclean/build/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/build/Debug/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/build/Release/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/out/Debug/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/Debug/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/out/Release/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/Release/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/build/default/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/compiled/20.13.1/linux/arm64/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/addon-build/release/install-root/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/addon-build/debug/install-root/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/addon-build/default/install-root/node_pqclean.node
  → /projects/DarkMessengerCrypto/node_modules/pqclean/lib/binding/node-v115-linux-arm64/node_pqclean.node

The warning triggers when calling:

await PQClean.kem.generateKeyPair('kyber1024');

The generateKeyPair method is unable to generate the keys:

{"publicKey":{},"privateKey":{}}

[StringManolo]

Sorry my mistake. I Didn't used properly the export method.

Here is the right way in case someone end's up here with same issue:

const { publicKey, privateKey } = await PQClean.kem.generateKeyPair('kyber1024');
const publicKeyHex = Buffer.from(publicKey.export()).toString('hex');
const privateKeyHex = Buffer.from(privateKey.export()).toString('hex');

• Also to avoid the warning in node, i use next line before the imports of the libs:

  process.removeAllListeners('warning');

If anyone needs an example of a full implementation of encryption/decryption: https://github.com/StringManolo/DarkMessengerCrypto/blob/6d6f63d53c49ae9505c85efa9c70255879f649e3/crypto_modules/KYBER_module.js

(Usually you do 2 way comunication for the shared secret. This module is for 1 way comunication, but you can tune it)

[tniessen]

@StringManolo Glad to hear it's working! Are you intentionally avoiding the native binding (and thus using the WebAssembly backend) or is there an issue that's causing your application to use the WebAssembly backend (and thus emit the warning)?

[StringManolo]

@tniessen i'm using the default config (pref-native):

npm run build-native

> pqclean@0.4.0 build-native
> node-gyp rebuild -j max

gyp info it worked if it ends with ok
gyp info using node-gyp@10.1.0
gyp info using node@20.13.1 | linux | arm64
gyp info find Python using Python version 3.12.3 found at "/usr/bin/python3"

gyp http GET https://nodejs.org/download/release/v20.13.1/node-v20.13.1-headers.tar.gz
gyp http fetch GET https://nodejs.org/download/release/v20.13.1/node-v20.13.1-headers.tar.gz attempt 1 failed with ETIMEDOUT
gyp WARN install got an error, rolling back install
gyp ERR! configure error
gyp ERR! stack FetchError: request to https://nodejs.org/download/release/v20.13.1/node-v20.13.1-headers.tar.gz failed, reason:
gyp ERR! stack at ClientRequest.<anonymous> (/usr/lib/node_modules/npm/node_modules/minipass-fetch/lib/index.js:130:14)
gyp ERR! stack at ClientRequest.emit (node:events:519:28)
gyp ERR! stack at _destroy (node:_http_client:880:13)
gyp ERR! stack at onSocketNT (node:_http_client:900:5)
gyp ERR! stack at process.processTicksAndRejections (node:internal/process/task_queues:83:21)
gyp ERR! System Linux 6.2.1-PRoot-Distro
gyp ERR! command "/usr/bin/node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild" "-j" "max"
gyp ERR! cwd /projects/DarkMessengerCrypto/node_modules/pqclean
gyp ERR! node -v v20.13.1
gyp ERR! node-gyp -v v10.1.0
gyp ERR! not ok

Is the download itself what is failling rigth ? Curl seems to be able to reach:

curl https://nodejs.org/download/release/v20.13.1/node-v20.13.1-headers.tar.gz -v
*   Trying 104.20.23.46:443...
*   Trying 2606:4700:10::6814:172e:443...
* Immediate connect fail for 2606:4700:10::6814:172e: Host is unreachable
*   Trying 2606:4700:10::6814:162e:443...
* Immediate connect fail for 2606:4700:10::6814:162e: Host is unreachable
* Connected to nodejs.org (104.20.23.46) port 443 (#0)
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=*.nodejs.org
*  start date: Feb 28 00:00:00 2024 GMT
*  expire date: Mar 30 23:59:59 2025 GMT
*  subjectAltName: host "nodejs.org" matched cert\'s "nodejs.org"
*  issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
*  SSL certificate verify ok.
> GET /download/release/v20.13.1/node-v20.13.1-headers.tar.gz HTTP/1.1
> Host: nodejs.org
> User-Agent: curl/7.82.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Wed, 10 Jul 2024 08:21:07 GMT
< Content-Type: application/gzip
< Content-Length: 8760725
< Connection: keep-alive
< last-modified: Thu, 09 May 2024 09:31:47 GMT
< etag: "663c9803-85ad95"
< x-robots-tag: noindex
< CF-Cache-Status: HIT
< Accept-Ranges: bytes
< Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
< X-Content-Type-Options: nosniff
< Server: cloudflare
< CF-RAY: 8a0f23705e4d6669-MAD
<
Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.
* Failure writing output to destination
* Closing connection 0
* TLSv1.3 (OUT), TLS alert, close notify (256):

File manually downloaded with curl:

$ du node-v20.13.1-headers.tar.gz -h
8.4M    node-v20.13.1-headers.tar.gz

$ md5sum node-v20.13.1-headers.tar.gz
5ac3eb8eb997be14965825e5d1e5b60e  node-v20.13.1-headers.tar.gz

Line 130 at /usr/lib/node_modules/npm/node_modules/minipass-fetch/lib/index.js is:

req.on('error', er => {
      // if a 'response' event is emitted before the 'error' event, then by the
      // time this handler is run it's too late to reject the Promise for the
      // response. instead, we forward the error event to the response stream
      // so that the error will surface to the user when they try to consume
      // the body. this is done as a side effect of aborting the request except
      // for in windows, where we must forward the event manually, otherwise
      // there is no longer a ref'd socket attached to the request and the
      // stream never ends so the event loop runs out of work and the process
      // exits without warning.
      // coverage skipped here due to the difficulty in testing
      // istanbul ignore next
      if (req.res) {
        req.res.emit('error', er)
      }
      reject(new FetchError(`request to ${request.url} failed, reason: ${
        er.message}`, 'system', er))
      finalize()
    })

[tniessen]

@StringManolo Thanks for looking into it. Unfortunately, these are not elements that I control.

The error message indicates a timeout, which might be related to node's odd Happy Eyeballs implementation (see, for example, https://github.com/nodejs/node/issues/52216 or https://github.com/orgs/nodejs/discussions/48028). Assuming this is the issue, as a workaround, could you try setting the environment variable NODE_OPTIONS=--no-enable-network-family-autoselection for the build process, and see if this resolves the download issue?

(Alternatively, maybe a more recent version of Node.js is less buggy.)

[StringManolo]

@tniessen Ty very much. Is built now and working.

The export NODE_OPTIONS=--no-enable-network-family-autoselection did the trick for downloading the file.

Also, after running npm run build-native got this error:

CC(target) Release/obj.target/pqclean_kem_mceliece348864/deps/PQClean/crypto_kem/mceliece348864/clean/crypto_int16.o
CC(target) Release/obj.target/pqclean_kem_mceliece348864/deps/PQClean/crypto_kem/mceliece348864/clean/crypto_int32.o
In file included from
../deps/PQClean/crypto_kem/mceliece348864/clean/controlbits.o] Error 1
../deps/PQClean/common/compat.h:20:21: error:
missing binary operator before token "("
20 | #  if !__GNUC_PREREQ(7, 1) // at least GCC 7.1
|                     ^
CC(target) Release/obj.target/pqclean_kem_mceliece348864/deps/PQClean/crypto_kem/mceliece348864/clean/crypto_uint16.o
make: *** [native/gen/pqclean_kem_mceliece348864.target.mk:121:
Release/obj.target/pqclean_kem_mceliece348864/deps/PQClean/crypto_kem/mceliece348864/clean/controlbits.o] Error 1
make: *** Waiting for unfinished jobs....
make: Leaving directory '/projects/DarkMessengerCrypto/node_modules/pqclean/build'
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack at ChildProcess.<anonymous> (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:209:23)
gyp ERR! System Linux 6.2.1-PRoot-Distro
gyp ERR! command "/usr/bin/node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild" "-j" "max"
gyp ERR! cwd /projects/DarkMessengerCrypto/node_modules/pqclean
gyp ERR! node -v v20.13.1
gyp ERR! node-gyp -v v10.1.0
gyp ERR! not ok

Fixed it by editing the deps/PQClean/common/compat.h and adding the missing definition:

#ifndef __GNUC_PREREQ
  #define __GNUC_PREREQ(maj, min) ((__GNUC__ << 16) + __GNUC_MINOR__ >= ((maj) << 16) + (min))
#endif

Added it just before this definition:

#if defined(__GNUC__) && !defined(__clang__)

I'm using this version of gcc:

gcc (Alpine 13.2.1_git20240309) 13.2.1 20240309
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.