Failed to make bytecode for arm64 on Linux x86

robvanderleek commented 1 month ago

What version of pkg are you using?

5.15.0

What version of Node.js are you using?

21.2.0

What operating system are you using?

Ubuntu

What CPU architecture are you using?

x86_64

What Node versions, OSs and CPU architectures are you building for?

node20-linuxstatic-arm64

Describe the Bug

Building a static binary for Linux ARM64 failes on Linux x86. For example, this command:

npx pkg -o index -t node20-linuxstatic-arm64 index.js

Generates the following error:

> pkg@5.15.0
> Warning Failed to make bytecode node20-arm64 for file /snapshot/yao-pkg-linuxstatic-arm64-bug/index.js

Adding the --debug flag to the build shows:

/home/rob/.pkg-cache/v3.5/fetched-v20.17.0-linuxstatic-arm64: 1: Syntax error: "(" unexpected
/home/rob/.pkg-cache/v3.5/fetched-v20.17.0-linuxstatic-arm64: 1: Syntax error: "(" unexpected
> Warning Failed to make bytecode node20-arm64 for file /snapshot/yao-pkg-linuxstatic-arm64-bug/index.js

I am able to generate an ARM64 static Linux binary on macOS.

Expected Behavior

A static Linux ARM64 is generated without warnings.

To Reproduce

I created a minimal example repository that demonstrates the problem:

git clone git@github.com:robvanderleek/yao-pkg-linuxstatic-arm64-bug.git

cd yao-pkg-linuxstatic-arm64-bug/
npm install
npx pkg -o index -t node20-linuxstatic-arm64 index.js

Not that:

npx pkg -o index -t node20-linuxstatic-x64 index.js

does not generate the warning.

robertsLando commented 1 month ago

@robvanderleek Could you tell me if it's woking with past versions of pkg or other nodejs versions like 18/16? I just want to understand where the issue could be

It should come from: https://github.com/yao-pkg/pkg/blob/db7f2c5ede385db7c29b7d17cf7fa605b44e9a9e/lib/fabricator.ts#L91

Could you debug it?

robvanderleek commented 1 month ago

Hi @robertsLando

See below how it fails for ARM64 with nodejs 20/18/16 versions, but works for x86 with nodejs 20/18/16.

I've tried pkg versions 5.15.0, 5.12.0, and 5.10.0, all failing.

I think it's host platform related, since building for arm64 fails on Linux Ubuntu 24, but succeeds on macOS.

rob:~/projects/yao-pkg-linuxstatic-arm64-bug$ npx pkg -o index -t node20-linuxstatic-arm64 index.js
> pkg@5.15.0
(node:19081) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
> Fetching base Node.js binaries to PKG_CACHE_PATH
  fetched-v20.17.0-linuxstatic-arm64  [====================] 100%

> Warning Failed to make bytecode node20-arm64 for file /snapshot/yao-pkg-linuxstatic-arm64-bug/index.js

rob:~/projects/yao-pkg-linuxstatic-arm64-bug$ npx pkg -o index -t node18-linuxstatic-arm64 index.js
> pkg@5.15.0
(node:19116) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
> Fetching base Node.js binaries to PKG_CACHE_PATH
  fetched-v18.20.4-linuxstatic-arm64  [====================] 100%

> Warning Failed to make bytecode node18-arm64 for file /snapshot/yao-pkg-linuxstatic-arm64-bug/index.js

rob:~/projects/yao-pkg-linuxstatic-arm64-bug$ npx pkg -o index -t node16-linuxstatic-arm64 index.js
> pkg@5.15.0
(node:19151) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
> Fetching base Node.js binaries to PKG_CACHE_PATH
  fetched-v16.20.2-linuxstatic-arm64  [====================] 100%

> Warning Failed to make bytecode node16-arm64 for file /snapshot/yao-pkg-linuxstatic-arm64-bug/index.js error (write EPIPE)

rob:~/projects/yao-pkg-linuxstatic-arm64-bug$ npx pkg -o index -t node20-linuxstatic-x64 index.js
> pkg@5.15.0
(node:19186) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
> Fetching base Node.js binaries to PKG_CACHE_PATH
  fetched-v20.17.0-linuxstatic-x64    [====================] 100%

  fetched-v20.17.0-linux-x64          [====================] 100%

rob:~/projects/yao-pkg-linuxstatic-arm64-bug$ npx pkg -o index -t node18-linuxstatic-x64 index.js
> pkg@5.15.0
(node:19224) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
> Fetching base Node.js binaries to PKG_CACHE_PATH
  fetched-v18.20.4-linuxstatic-x64    [====================] 100%

  fetched-v18.20.4-linux-x64          [====================] 100%

rob:~/projects/yao-pkg-linuxstatic-arm64-bug$ npx pkg -o index -t node16-linuxstatic-x64 index.js
> pkg@5.15.0
(node:19261) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
> Fetching base Node.js binaries to PKG_CACHE_PATH
  fetched-v16.20.2-linuxstatic-x64    [====================] 100%

  fetched-v16.20.2-linux-x64          [====================] 100%

robertsLando commented 1 month ago

@robvanderleek I actually have no time to look at this issue but I suggest to try debug it to see if you are able to find the root cause of the issue, aboce I linked you the code that makes the bytecode, try to put some breakpoints there to see what's the line that triggers that issue

thisjt commented 1 month ago

I'm able to replicate the issue in WSL (Ubuntu). This might be useful, I think.

I still have no idea where this Syntax error: "(" unexpected is coming from, as on my debug I think this is run asynchronously somewhere.

robertsLando commented 1 month ago

That comes from fabricator fabricate method that spaws the script you find on top to create the bytecode of the file

yao-pkg / pkg