yao-pkg / pkg-binaries

Collection of pkg nodejs binaries that are not supported by pkg
MIT License
89 stars 23 forks source link

cross compilation error #1

Closed serra82 closed 5 years ago

serra82 commented 5 years ago

I'm trying to cross compile from x64 to arm64 a simple hello world:
~/test$ cat hello.js
console.log('hello');

but I have the following error:
~/test$ pkg -t node8-linux-arm64 hello.js

pkg@4.3.1
Warning Failed to make bytecode node8-arm64 for file /snapshot/test/hello.js

the resulting binary not start on arm64 board.

robertsLando commented 5 years ago

Does it works if you try to compile it from the arm64 board?

serra82 commented 5 years ago

Does it works if you try to compile it from the arm64 board?

yes

robertsLando commented 5 years ago

This is not a problem related to my binary file, this is a know issue. You need the interpreter for the binary file. Check this issue on pkg:

https://github.com/zeit/pkg/issues/310

Check the @Huerlisi answer.

You are using an x64 system so you need to install interpreter for arm64.

Try this:

sudo dpkg --add-architecture arm64

Than try again to compile. Let me know :)

serra82 commented 5 years ago

sudo dpkg --add-architecture arm64

this is not sufficient.

I also tried:

but these packets aren't available.
So I didn't resolve the problem yet.

robertsLando commented 5 years ago

If you run file command with node binary you get this output:

/pkg-binaries/arm64$ file fetched-v8.11.3-linux-arm64 
fetched-v8.11.3-linux-arm64: ELF 64-bit LSB shared object, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=02bf3444ecc520c4da40e89cbfbf6831e3a205ea, with debug_info, not stripped

As mentioned here to use this binary you need the /lib/ld-linux-aarch64.so.1 interpreter on your device.

Try this tutorial: https://chrislea.com/2018/08/20/cross-compiling-node-js-for-arm-on-ubuntu/ (At the end there is also explained the 64 bit).

In few words:

sudo dpkg --add-architecture arm64
sudo apt install binutils-aarch64-linux-gnu \
> gcc-aarch64-linux-gnu \
> g++-aarch64-linux-gnu

Here you can find another tutorial: http://www.pilotlogic.com/sitejoom/index.php/forum/cross-build-development/3937-cross-compile-from-intel-64-linux-to-arm-64-linux

Remember also that (from https://github.com/zeit/pkg#native-addons):

When a package, that contains a native module, is being installed, the native module is compiled against current system-wide Node.js version. Then, when you compile your project with pkg, pay attention to --target option. You should specify the same Node.js version as your system-wide Node.js to make compiled executable compatible with .node files.

With the script in utils folder in this repo .node files are automatically added to the pkg folder with the executable generated by pkg but you must be sure that you are running the same nodejs version used to compile the pkg bin or the .node files will not work. If nodejs version are different you should use nvm to swith to the same nodejs version and run the comand npm rebuild to rebuild all dependencies with the new version installed. This is not your case because you are compiling a simple project but for other project this problem also could happen.

I also suggest you to use option --debug with pkg to have more details about your error.

robertsLando commented 5 years ago

Closing this after no more feedback

PerminovEugene commented 3 years ago

@robertsLando I have an error /home/eugeneperminov/.pkg-cache/v2.6/fetched-v12.18.1-linux-arm64: 1: /home/eugeneperminov/.pkg-cache/v2.6/fetched-v12.18.1-linux-arm64: Syntax error: ")" unexpected /home/eugeneperminov/.pkg-cache/v2.6/fetched-v12.18.1-linux-arm64: 1: /home/eugeneperminov/.pkg-cache/v2.6/fetched-v12.18.1-linux-arm64: Syntax error: ")" unexpected events.js:292 throw er; // Unhandled 'error' event ^

Error [ERR_STREAM_DESTROYED]: Cannot call write after a stream was destroyed at doWrite (_stream_writable.js:399:19)

Js file has only 1 line - console.log('test'); I am using v12.18.1-linux-arm64 from your binaries. I've checked that system node version is the same and chmod on js file and I installed all librares which you mentoined above. Do you have any suggestions?

robertsLando commented 3 years ago

Could you paste here the commands you are using?

PerminovEugene commented 3 years ago

@robertsLando pkg ./test.js -t node12.18.1-linux-arm64 --debug

PerminovEugene commented 3 years ago

What I 've installed: sudo dpkg --add-architecture arm64 sudo apt-get update sudo apt-get install -y libc6:arm64 libstdc++6:arm64 sudo apt install binutils-aarch64-linux-gnu \ sudo apt install gcc-aarch64-linux-gnu \ sudo apt installg++-aarch64-linux-gnu

robertsLando commented 3 years ago

What's the output of uname -a command?

robertsLando commented 3 years ago

Anyway you should use: pkg test.js -t node12-linux-arm64

PerminovEugene commented 3 years ago

@robertsLando uname -a Linux eugene 5.4.0-47-generic #51~18.04.1-Ubuntu SMP Sat Sep 5 14:35:50 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

pkg ./test.js -t node12-linux-arm64 -d - has the same result

robertsLando commented 3 years ago

You cannot build arm64 binaries on an amd64 (x86_64) machine.

PerminovEugene commented 3 years ago

Which machine I need? Would docker image be ok?

robertsLando commented 3 years ago

Sorry that is true only if you also have some node modules to bind with your executable,anyway what I don't understand is if that error is throw in compilation time or when you execute the compiled binary?

PerminovEugene commented 3 years ago

No, it happens when I try to pack the only js file without any dependencies with command pkg test.js -t node12-linux-arm64

PerminovEugene commented 3 years ago

Does this binary work for you?

robertsLando commented 3 years ago

@PerminovEugene Yes, it works

robertsLando commented 3 years ago

You could try to use docker buildx to test in an arm64 env. Just download the binary, place it inside the pkg-fetch folder, compile and execute the compiled bin

PerminovEugene commented 3 years ago

@robertsLando I can't build only use binaries from this library. Does docker buildx can help with this? I can't even find correct docker image with arm64 , linux and node. If you have this could you share, please?

PerminovEugene commented 3 years ago

@robertsLando I made a minimal repo with docker image with required installed tools and it reproduces my error with streams https://github.com/PerminovEugene/arm64pkg Could you have a look, please?

robertsLando commented 3 years ago

Dockerfile:

FROM node:12.18.1

RUN apt-get update && apt-get install -y \
    build-essential \
    && rm -rf /var/lib/apt/lists/*

RUN npm i pkg -g --unsafe-perm

COPY ./test.js .
COPY ./fetched-v12.18.1-linux-arm64 .

RUN mkdir -p "$HOME/.pkg-cache/v2.6"

RUN cp fetched-v12.18.1-linux-arm64 "$HOME/.pkg-cache/v2.6"

CMD pkg test.js -t node12-linux-arm64

build script

#!/bin/bash

docker buildx build -f Dockerfile --platform linux/arm64/v8 \
    -t yao-pkg/test:arm64-latest \
    --load \
    .

In this way, it works on my machine.

thor027 commented 3 years ago

1.I have also get this problem,my envirmonent : nodejs-12.18.1 pkg-4.4.9 system info:

uname -a

Linux host-192-168-0-205 4.19.90-17.ky10.aarch64 #1 SMP Sun Jun 28 14:27:40 CST 2020 aarch64 aarch64 aarch64 GNU/Linux

2.

file /root/.pkg-cache/v2.6/fetched-v12.18.1-linux-arm64

/root/.pkg-cache/v2.6/fetched-v12.18.1-linux-arm64: ELF 64-bit LSB shared object, ARM aarch64, version 1 (GNU/Linux), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, for GNU/Linux 3.7.0, BuildID[sha1]=c3fde698f9954ae7f2f95307764414de185f3838, with debug_info, not stripped

3.

dpkg --add-architecture arm64

apt-get update

-bash: apt-get: command not found

yum install apt-get

Last metadata expiration check: 2:59:30 ago on Fri 23 Oct 2020 02:35:01 PM CST. No match for argument: apt-get Error: Unable to find a match: apt-get

4. in my nodejs project httpget.js、node_modules、package.json

node httpget.js --- is OK

/root/.pkg-cache/v2.6/fetched-v12.18.1-linux-arm64

<--- Last few GCs ---> <--- JS stacktrace ---> #

Fatal process OOM in insufficient memory to create an Isolate

-- is OK,but execute result is empty,nothing useful.

5. get to do below is error!

pkg -t linux package.json

pkg@4.4.9 Warning Failed to make bytecode node12-arm64 for file /snapshot/VectorTiles/httpget.js Warning Failed to make bytecode node12-arm64 for file /snapshot/VectorTiles/node_modules/async/dist/async.js

it create empty project package,no js execute result. please help me get it, thank you very much!

if you will want to test this problem,I can give you vm machine acount.