search

source-academy / js-slang

Implementations of sublanguages of JavaScript, TypeScript, Scheme and Python
https://source-academy.github.io/source/
Apache License 2.0
66 stars 103 forks source link

Cannot npm install due to gpu.js/angle not compiling #1417

Open LittleLightLittleFire opened 1 year ago

LittleLightLittleFire commented 1 year ago 
$  npm install js-slang

npm ERR! code 1
npm ERR! path /xxxx/xxxx/node_modules/gpu.js/node_modules/gl
npm ERR! command failed
npm ERR! command sh -c prebuild-install || node-gyp rebuild
npm ERR! make: Entering directory '/xxxx/xxxx/node_modules/gpu.js/node_modules/gl/build'
npm ERR!   CXX(target) Release/obj.target/angle_common/angle/src/common/Float16ToFloat32.o
npm ERR! make: Leaving directory '/xxxx/xxxx/node_modules/gpu.js/node_modules/gl/build'
npm ERR! prebuild-install warn install No prebuilt binaries found (target=18.16.0 runtime=node arch=x64 libc= platform=linux)
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@9.3.1
npm ERR! gyp info using node@18.16.0 | linux | x64
npm ERR! gyp info find Python using Python version 3.11.3 found at "/usr/bin/python3"
npm ERR! gyp info spawn /usr/bin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   '/xxxx/xxxx/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'make',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/xxxx/xxxx/node_modules/gpu.js/node_modules/gl/build/config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/xxxx/xxxx/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/xxxx/xxxx/.cache/node-gyp/18.16.0/include/node/common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=/xxxx/xxxx/.cache/node-gyp/18.16.0',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=/xxxx/xxxx/node_modules/node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=/xxxx/xxxx/.cache/node-gyp/18.16.0/<(target_arch)/node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=/xxxx/xxxx/node_modules/gpu.js/node_modules/gl',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp info spawn make
npm ERR! gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
npm ERR! In file included from ../angle/src/common/debug.h:16,
npm ERR!                  from ../angle/src/common/mathutil.h:12,
npm ERR!                  from ../angle/src/common/Float16ToFloat32.cpp:9:
npm ERR! ../angle/src/common/angleutils.h:36:14: error: ‘uintptr_t’ does not name a type
npm ERR!    36 | extern const uintptr_t DirtyPointer;
npm ERR!       |              ^~~~~~~~~
npm ERR! ../angle/src/common/angleutils.h:21:1: note: ‘uintptr_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
npm ERR!    20 | #include <vector>
npm ERR!   +++ |+#include <cstdint>
npm ERR!    21 | 
npm ERR! make: *** [angle/src/angle_common.target.mk:159: Release/obj.target/angle_common/angle/src/common/Float16ToFloat32.o] Error 1
npm ERR! gyp ERR! build error 
npm ERR! gyp ERR! stack Error: `make` failed with exit code: 2
npm ERR! gyp ERR! stack     at ChildProcess.onExit (/xxxx/xxxx/node_modules/node-gyp/lib/build.js:203:23)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
npm ERR! gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:291:12)
npm ERR! gyp ERR! System Linux 6.3.1-arch1-1
npm ERR! gyp ERR! command "/xxxx/xxxx/.nvm/versions/node/v18.16.0/bin/node" "/xxxx/xxxx/node_modules/.bin/node-gyp" "rebuild"
npm ERR! gyp ERR! cwd /xxxx/xxxx/node_modules/gpu.js/node_modules/gl
npm ERR! gyp ERR! node -v v18.16.0
npm ERR! gyp ERR! node-gyp -v v9.3.1
npm ERR! gyp ERR! not ok
RichDom2185 commented 1 year ago

Hi! Do you mind providing the following information for troubleshooting:

Thanks!

LittleLightLittleFire commented 1 year ago

Arch Linux x64 v18.16.0

LittleLightLittleFire commented 1 year ago

Docker file for easy repo:

FROM archlinux/archlinux:base-devel

RUN pacman -Syu npm xorg --noconfirm
RUN npm install -g js-slang
Step 3/3 : RUN npm install -g js-slang
 ---> Running in 0987d79bcc28
npm WARN deprecated @npmcli/move-file@2.0.1: This functionality has been moved to @npmcli/fs
npm notice 
npm notice New major version of npm available! 8.19.2 -> 9.6.6
npm notice Changelog: <https://github.com/npm/cli/releases/tag/v9.6.6>
npm notice Run `npm install -g npm@9.6.6` to update!
npm notice 
npm ERR! code 1
npm ERR! path /usr/lib/node_modules/js-slang/node_modules/gl
npm ERR! command failed
npm ERR! command sh -c -- prebuild-install || node-gyp rebuild
npm ERR! make: Entering directory '/usr/lib/node_modules/js-slang/node_modules/gl/build'
npm ERR!   CXX(target) Release/obj.target/angle_common/angle/src/common/Float16ToFloat32.o
npm ERR! make: Leaving directory '/usr/lib/node_modules/js-slang/node_modules/gl/build'
npm ERR! prebuild-install warn install No prebuilt binaries found (target=19.9.0 runtime=node arch=x64 libc= platform=linux)
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using node-gyp@9.3.1
npm ERR! gyp info using node@19.9.0 | linux | x64
npm ERR! gyp info find Python using Python version 3.11.3 found at "/usr/sbin/python3"
npm ERR! gyp http GET https://nodejs.org/download/release/v19.9.0/node-v19.9.0-headers.tar.gz
npm ERR! gyp http 200 https://nodejs.org/download/release/v19.9.0/node-v19.9.0-headers.tar.gz
npm ERR! gyp http GET https://nodejs.org/download/release/v19.9.0/SHASUMS256.txt
npm ERR! gyp http 200 https://nodejs.org/download/release/v19.9.0/SHASUMS256.txt
npm ERR! gyp info spawn /usr/sbin/python3
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args   '/usr/lib/node_modules/js-slang/node_modules/node-gyp/gyp/gyp_main.py',
npm ERR! gyp info spawn args   'binding.gyp',
npm ERR! gyp info spawn args   '-f',
npm ERR! gyp info spawn args   'make',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/usr/lib/node_modules/js-slang/node_modules/gl/build/config.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/usr/lib/node_modules/js-slang/node_modules/node-gyp/addon.gypi',
npm ERR! gyp info spawn args   '-I',
npm ERR! gyp info spawn args   '/root/.cache/node-gyp/19.9.0/include/node/common.gypi',
npm ERR! gyp info spawn args   '-Dlibrary=shared_library',
npm ERR! gyp info spawn args   '-Dvisibility=default',
npm ERR! gyp info spawn args   '-Dnode_root_dir=/root/.cache/node-gyp/19.9.0',
npm ERR! gyp info spawn args   '-Dnode_gyp_dir=/usr/lib/node_modules/js-slang/node_modules/node-gyp',
npm ERR! gyp info spawn args   '-Dnode_lib_file=/root/.cache/node-gyp/19.9.0/<(target_arch)/node.lib',
npm ERR! gyp info spawn args   '-Dmodule_root_dir=/usr/lib/node_modules/js-slang/node_modules/gl',
npm ERR! gyp info spawn args   '-Dnode_engine=v8',
npm ERR! gyp info spawn args   '--depth=.',
npm ERR! gyp info spawn args   '--no-parallel',
npm ERR! gyp info spawn args   '--generator-output',
npm ERR! gyp info spawn args   'build',
npm ERR! gyp info spawn args   '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp info spawn make
npm ERR! gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
npm ERR! In file included from ../angle/src/common/debug.h:16,
npm ERR!                  from ../angle/src/common/mathutil.h:12,
npm ERR!                  from ../angle/src/common/Float16ToFloat32.cpp:9:
npm ERR! ../angle/src/common/angleutils.h:36:14: error: ‘uintptr_t’ does not name a type
npm ERR!    36 | extern const uintptr_t DirtyPointer;
npm ERR!       |              ^~~~~~~~~
npm ERR! ../angle/src/common/angleutils.h:21:1: note: ‘uintptr_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
npm ERR!    20 | #include <vector>
npm ERR!   +++ |+#include <cstdint>
npm ERR!    21 | 
npm ERR! make: *** [angle/src/angle_common.target.mk:157: Release/obj.target/angle_common/angle/src/common/Float16ToFloat32.o] Error 1
npm ERR! gyp ERR! build error 
npm ERR! gyp ERR! stack Error: `make` failed with exit code: 2
npm ERR! gyp ERR! stack     at ChildProcess.onExit (/usr/lib/node_modules/js-slang/node_modules/node-gyp/lib/build.js:203:23)
npm ERR! gyp ERR! stack     at ChildProcess.emit (node:events:513:28)
npm ERR! gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:293:12)
npm ERR! gyp ERR! System Linux 6.3.1-arch1-1
npm ERR! gyp ERR! command "/usr/bin/node" "/usr/lib/node_modules/js-slang/node_modules/.bin/node-gyp" "rebuild"
npm ERR! gyp ERR! cwd /usr/lib/node_modules/js-slang/node_modules/gl
npm ERR! gyp ERR! node -v v19.9.0
npm ERR! gyp ERR! node-gyp -v v9.3.1
npm ERR! gyp ERR! not ok
RichDom2185 commented 1 year ago

I can reproduce this running via Docker on an M1 Mac (using the --platform=linux/amd64 switch). Let us look into it.

Thanks for reporting!

RichDom2185 commented 1 year ago

Sorry for the delay; I strongly suspect it has something to do with the Python version (3.11) - it is a known issue that node-gyp is not compatible with Python 3.11. I am still trying to install Python 3.10 manually, but without success (not too familiar using arch/yay unfortunately). I'm trying out a couple different configs.

Is using Arch a necessary requirement?

RichDom2185 commented 1 year ago

Update: figured out how to get Python 3.10 set up, but it still did not resolve the issue:

FROM --platform=linux/amd64 archlinux/archlinux:base-devel

# Makepkg requires non-root user
RUN useradd --create-home --shell=/bin/false build && usermod -L build
RUN echo "build ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers

# Install yay
RUN pacman -Syu git --noconfirm
RUN git clone https://aur.archlinux.org/yay-bin.git /tmp/yay-bin
RUN chown -R build:build /tmp/yay-bin
WORKDIR /tmp/yay-bin
RUN sudo -u build makepkg --noconfirm -si

# Install python 3.10
RUN sudo -u build yay -S python310 --noconfirm

# Install js-slang
RUN pacman -Sy npm xorg --noconfirm
RUN npm install --python=python3.10 -g js-slang

results in the same error.

RichDom2185 commented 1 year ago

Unfortunately, I still could not get it to run on Arch, but Alpine as well as Debian/Ubuntu works, although a global install will result to a module resolution error (something to do with our py-slang submodule), thus we need to build from source:

FROM nikolaik/python-nodejs:python3.10-nodejs16-alpine

RUN apk update && apk add bash alpine-sdk pkgconfig libx11-dev libxi-dev libxext-dev

WORKDIR /usr/src
RUN git clone --recurse-submodules --depth 1 https://github.com/source-academy/js-slang.git

WORKDIR /usr/src/js-slang
# Yarn is preinstalled
RUN yarn install --frozen-lockfile
RUN yarn build

# Add to PATH (following README instructions)
WORKDIR /usr/src/js-slang/dist
RUN npm link

ENTRYPOINT [ "js-slang" ]
LittleLightLittleFire commented 1 year ago

I've had a deeper look and it is not an issue with python 3.11 It seems like the bug with is the gl​ package (which is a dependncy of js-slang)

e.g. changing it to npm install gl does not work either. The simple reason is that the gl package (https://github.com/stackgl/headless-gl) uses a super old version of angle, which has a minor bug where it uses uintptr without importing #include <cstdint>

The dependency tree is as follows js-slang -> gpu.js -> gl

RichDom2185 commented 1 year ago

The simple reason is that the gl package (https://github.com/stackgl/headless-gl) uses a super old version of angle, which has a minor bug where it uses uintptr without importing #include <cstdint>

I realise this is the issue as well. It used to be allowed in GCC versions below 13, but not anymore.

Setting GCC version to 12 works (still have to build from source, though):

FROM --platform=linux/amd64 archlinux/archlinux:base-devel

RUN pacman -Syu git npm xorg gcc12 --noconfirm
RUN npm install -g yarn

WORKDIR /usr/src
RUN git clone --recurse-submodules --depth 1 https://github.com/source-academy/js-slang.git
WORKDIR /usr/src/js-slang
RUN CC=gcc-12 CXX=g++-12 yarn install --frozen-lockfile
RUN yarn build

WORKDIR /usr/src/js-slang/dist
RUN npm link

ENTRYPOINT [ "js-slang" ]
LittleLightLittleFire commented 1 year ago

This is fine, I can work with this.

LittleLightLittleFire commented 1 year ago

I've gotten it to build with:

FROM archlinux/archlinux:base-devel

RUN pacman -Syu npm xorg gcc12 --noconfirm
RUN CC=gcc-12 CXX=g++-12 npm install -g js-slang

However when I run it:

docker tag d15a212f835c test
docker run -ti test bash
[root@9a0ed9a26213 /]# js-slang 
node:internal/modules/cjs/loader:1051
  throw err;
  ^

Error: Cannot find module 'fast-levenshtein'
Require stack:
- /usr/lib/node_modules/js-slang/dist/py-slang/src/resolver.js
- /usr/lib/node_modules/js-slang/dist/py-slang/src/index.js
- /usr/lib/node_modules/js-slang/dist/parser/python/index.js
- /usr/lib/node_modules/js-slang/dist/parser/parser.js
- /usr/lib/node_modules/js-slang/dist/stdlib/parser.js
- /usr/lib/node_modules/js-slang/dist/stdlib/index.js
- /usr/lib/node_modules/js-slang/dist/modules/requireProvider.js
- /usr/lib/node_modules/js-slang/dist/modules/moduleLoader.js
- /usr/lib/node_modules/js-slang/dist/interpreter/interpreter.js
- /usr/lib/node_modules/js-slang/dist/interpreter/closure.js
- /usr/lib/node_modules/js-slang/dist/utils/stringify.js
- /usr/lib/node_modules/js-slang/dist/errors/errors.js
- /usr/lib/node_modules/js-slang/dist/schedulers.js
- /usr/lib/node_modules/js-slang/dist/createContext.js
- /usr/lib/node_modules/js-slang/dist/index.js
- /usr/lib/node_modules/js-slang/dist/repl/repl.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1048:15)
    at Module._load (node:internal/modules/cjs/loader:901:27)
    at Module.require (node:internal/modules/cjs/loader:1115:19)
    at require (node:internal/modules/helpers:119:18)
    at Object.<anonymous> (/usr/lib/node_modules/js-slang/dist/py-slang/src/resolver.js:7:21)
    at Module._compile (node:internal/modules/cjs/loader:1233:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1287:10)
    at Module.load (node:internal/modules/cjs/loader:1091:32)
    at Module._load (node:internal/modules/cjs/loader:938:12)
    at Module.require (node:internal/modules/cjs/loader:1115:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/usr/lib/node_modules/js-slang/dist/py-slang/src/resolver.js',
    '/usr/lib/node_modules/js-slang/dist/py-slang/src/index.js',
    '/usr/lib/node_modules/js-slang/dist/parser/python/index.js',
    '/usr/lib/node_modules/js-slang/dist/parser/parser.js',
    '/usr/lib/node_modules/js-slang/dist/stdlib/parser.js',
    '/usr/lib/node_modules/js-slang/dist/stdlib/index.js',
    '/usr/lib/node_modules/js-slang/dist/modules/requireProvider.js',
    '/usr/lib/node_modules/js-slang/dist/modules/moduleLoader.js',
    '/usr/lib/node_modules/js-slang/dist/interpreter/interpreter.js',
    '/usr/lib/node_modules/js-slang/dist/interpreter/closure.js',
    '/usr/lib/node_modules/js-slang/dist/utils/stringify.js',
    '/usr/lib/node_modules/js-slang/dist/errors/errors.js',
    '/usr/lib/node_modules/js-slang/dist/schedulers.js',
    '/usr/lib/node_modules/js-slang/dist/createContext.js',
    '/usr/lib/node_modules/js-slang/dist/index.js',
    '/usr/lib/node_modules/js-slang/dist/repl/repl.js'
  ]
}

Node.js v20.5.0
[root@9a0ed9a26213 /]#
RichDom2185 commented 1 year ago

I encountered the same issue when I tried to install globally using -g; hence why in the above examples that I gave, I built js-slang from source instead (and subsequently set the entrypoint in the Dockerfile).

I discussed offline with @Fidget-Spinner and he suggested that the py-slang submodule might not be getting TS-compiled when installing js-slang using -g, but we haven't looked into it yet. @Fidget-Spinner could you advice on this?

LittleLightLittleFire commented 1 year ago

I've done a source install as above, and it does run but it shows this message

[build@7149f3b17375 ~]$ js-slang                                                                                                                                                                                               
(node:7) [DEP0128] DeprecationWarning: Invalid 'main' field in '/opt/js-slang/node_modules/node-getopt/package.json' of './lib'. Please either fix that or report it to the module author                                       
(Use `node --trace-deprecation ...` to show where the warning was created)                                                                                                                                                       
undefined                                                                                                                                                                                                                        
>                                                                                                                                                                                                                                
[build@7149f3b17375 ~]$

I've patched it with

RUN sudo sed -e 's#"main" : "./lib",##' -i /usr/lib/node_modules/js-slang/node_modules/node-getopt/package.json
Fidget-Spinner commented 1 year ago

I encountered the same issue when I tried to install globally using -g; hence why in the above examples that I gave, I built js-slang from source instead (and subsequently set the entrypoint in the Dockerfile).

I discussed offline with @Fidget-Spinner and he suggested that the py-slang submodule might not be getting TS-compiled when installing js-slang using -g, but we haven't looked into it yet. @Fidget-Spinner could you advice on this?

Oops, now that I see the full log posted here, it seems that it's because we don't include the dependencies for py-slang and scm-slang in js-slang's package.json. The only reason scm-slang works is that coincidentally all the packages it needs is also installed by js-slang. but once scm-slang adds a separate dependency, the build will fail.

The solution it seems, is to install py-slang's dependencies into js-slang.

JLanzuisi commented 9 months ago

Hi! Also getting this issue in Ubuntu 23.10. Is there any workaround? Im comming from npm and get this when doing yarn add sicp.

edit: I worked around the issue using nix-shell. This is the shell.nix, maybe it'll be useful for someone else:

{ pkgs ? import <nixpkgs> {} }:
pkgs.mkShell { 
  buildInputs = [
    pkgs.yarn
    pkgs.nodejs_18
    pkgs.python3
    pkgs.libGL
    pkgs.xorg.libX11
    pkgs.xorg.libXext
    pkgs.xorg.libXi
  ];
}
RichDom2185 commented 9 months ago

Hi! Also getting this issue in Ubuntu 23.10. Is there any workaround? Im comming from npm and get this when doing yarn add sicp.

Hi, was trying to reproduce the issue; sorry it took a while, it seems you've solved it which is great!

I was able to reproduce it inside a ubuntu:23.10 Docker container and it seems that the root cause it similar to the above: it ships with GCC version 13 by default, which came with this breaking change.

The solution is just to install and use GCC 12:

apt install -y gcc-12 g++-12

You also need some additional dependencies (if your environment does not already have it, e.g. test/GitHub Actions runner environment):

apt install -y pkg-config libxi-dev libgl1-mesa-dev

Then, to add the dependency:

CC=gcc-12 CXX=g++-12 yarn add sicp

Full setup commands (starting from a blank ubuntu:23.10 Docker container (note that I'm using Node 20.9 installed via nvm and Python 3.11.3 for testing):

cd
apt update && apt upgrade -y
apt install -y curl
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.5/install.sh | bash
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"
nvm install 20.9
npm i -g yarn
apt install -y gcc-12 g++-12 python3 python3-pip python-is-python3
apt install -y pkg-config libxi-dev libgl1-mesa-dev
mkdir testing
cd testing
npm init -y
CC=gcc-12 CXX=g++-12 yarn add sicp