search

sagemathinc / cowasm

CoWasm: Collaborative WebAssembly for Servers and Browsers. Built using Zig. Supports Python with extension modules, including numpy.
https://cowasm.org
BSD 3-Clause "New" or "Revised" License
491 stars 25 forks source link

Build fails on Arch Linux #53

Closed oroppas closed 1 year ago

oroppas commented 1 year ago

Hi,

I haven't been able to figure out why the build fails on my side. In the log:

LOG from webpack.FileSystemInfo
<w> Managed item /home/ryutas/packages/cowasm/cowasm/packages/browser/node_modules/python-wasm isn't a directory or doesn't contain a package.json (see snapshot.managedPaths option)
+ 12 hidden lines

and

ryutas@archlinux ~/packages/cowasm/cowasm
ls -l packages/browser/node_modules/python-wasm
lrwxrwxrwx 1 ryutas ryutas 17 Jan 20 07:16 packages/browser/node_modules/python-wasm -> ../../python-wasm

but I don't see any pytnon-wasm in packages directory. Am I missing something? Any advice will be greatly appreciated. Thank you!

Here is the full log:

ryutas@archlinux ~/packages/cowasm/cowasm
make
./bin/make-all all packages/browser/ packages/build/ packages/bzip2/ packages/coreutils/ packages/cowasm.org/ packages/cowasm.sh/ packages/dash/ packages/dash-wasm/ packages/dylink/ packages/f2c/ packages/kernel/ packages/less/ packages/libedit/ packages/libffi/ packages/libgit2/ packages/libpng/ packages/lua/ packages/luatex/ packages/lzma/ packages/man/ packages/ncurses/ packages/openssl/ packages/posix-browser/ packages/posix-node/ packages/posix-wasm/ packages/rogue/ packages/sqlite/ packages/tar/ packages/termcap/ packages/terminal/ packages/viz/ packages/wasi-js/ packages/wasm-opt/ packages/zlib/

TARGET=$1
shift
ROOTDIR=`pwd`

for PACKAGE in $*
do
    cd "$ROOTDIR/$PACKAGE"
    echo $PACKAGE
    if [ $TARGET = "all" ]; then
        make
    else
        make $TARGET
    fi
done
packages/browser/
make[1]: Entering directory '/home/ryutas/packages/cowasm/cowasm/packages/browser'
npm ci

added 471 packages, and audited 1886 packages in 12s

102 packages are looking for funding
  run `npm fund` for details

1 high severity vulnerability

To address all issues, run:
  npm audit fix

Run `npm audit` for details.
npm run build

> @cowasm/browser@0.3.0 build
> webpack

asset main.bundle.js 461 KiB [compared for emit] (name: main)
asset index.html 246 bytes [compared for emit]
runtime modules 1.13 KiB 5 modules
modules by path ./node_modules/ 156 KiB
  modules by path ./node_modules/assert/build/ 67.8 KiB 4 modules
  modules by path ./node_modules/object-is/*.js 1.14 KiB 4 modules
  modules by path ./node_modules/is-nan/*.js 1.2 KiB 4 modules
  modules by path ./node_modules/util/ 27.9 KiB 3 modules
  modules by path ./node_modules/object-keys/*.js 4.36 KiB 3 modules
  modules by path ./node_modules/debug/src/*.js 12 KiB 2 modules
  modules by path ./node_modules/call-bind/*.js 1.68 KiB 2 modules
  modules by path ./node_modules/function-bind/*.js 1.55 KiB 2 modules
  modules by path ./node_modules/has-symbols/*.js 2.13 KiB 2 modules
  + 18 modules
modules by path ./src/*.ts 9.31 KiB
  ./src/index.ts 2.94 KiB [built] [code generated]
  ./src/python.ts 6.37 KiB [built] [code generated] [1 error]

LOG from webpack.FileSystemInfo
<w> Managed item /home/ryutas/packages/cowasm/cowasm/packages/browser/node_modules/python-wasm isn't a directory or doesn't contain a package.json (see snapshot.managedPaths option)
+ 12 hidden lines

ERROR in ./src/python.ts 38:0-37
Module not found: Error: Can't resolve 'python-wasm' in '/home/ryutas/packages/cowasm/cowasm/packages/browser/src'
 @ ./src/index.ts 39:0-30 46:37-43

ERROR in /home/ryutas/packages/cowasm/cowasm/packages/browser/src/kernel.ts
3:28-44
[tsl] ERROR in /home/ryutas/packages/cowasm/cowasm/packages/browser/src/kernel.ts(3,29)
      TS2306: File '/home/ryutas/packages/cowasm/cowasm/packages/kernel/index.d.ts' is not a module.

ERROR in /home/ryutas/packages/cowasm/cowasm/packages/browser/src/python.ts
./src/python.ts 3:23-36
[tsl] ERROR in /home/ryutas/packages/cowasm/cowasm/packages/browser/src/python.ts(3,24)
      TS2307: Cannot find module 'python-wasm' or its corresponding type declarations.
 @ ./src/index.ts 39:0-30 46:37-43

3 errors have detailed information that is not shown.
Use 'stats.errorDetails: true' resp. '--stats-error-details' to show it.

webpack 5.73.0 compiled with 3 errors in 1720 ms
make[1]: *** [Makefile:9: /home/ryutas/packages/cowasm/cowasm/packages/browser/dist/.built] Error 1
make[1]: Leaving directory '/home/ryutas/packages/cowasm/cowasm/packages/browser'
make: *** [Makefile:10: all] Error 2
williamstein commented 1 year ago

Could you give the exact steps you take, from start to finish, starting with "git clone ..."?

oroppas commented 1 year ago

Here's the exact steps:

$ git clone --recursive https://github.com/sagemathinc/cowasm
$ cd cowasm
$ make

and the full log:

ryutas@archlinux ~/packages/cowasm
git clone --recursive https://github.com/sagemathinc/cowasm
Cloning into 'cowasm'...
remote: Enumerating objects: 20441, done.
remote: Counting objects: 100% (1380/1380), done.
remote: Compressing objects: 100% (515/515), done.
remote: Total 20441 (delta 895), reused 1308 (delta 840), pack-reused 19061
Receiving objects: 100% (20441/20441), 13.33 MiB | 9.37 MiB/s, done.
Resolving deltas: 100% (14775/14775), done.

ryutas@archlinux ~/packages/cowasm
cd cowasm/

ryutas@archlinux ~/packages/cowasm/cowasm
make
./bin/make-all all packages/browser/ packages/build/ packages/bzip2/ packages/coreutils/ packages/cowasm.org/ packages/cowasm.sh/ packages/dash/ packages/dash-wasm/ packages/dylink/ packages/f2c/ packages/kernel/ packages/less/ packages/libedit/ packages/libffi/ packages/libgit2/ packages/libpng/ packages/lua/ packages/luatex/ packages/lzma/ packages/man/ packages/ncurses/ packages/openssl/ packages/posix-browser/ packages/posix-node/ packages/posix-wasm/ packages/rogue/ packages/sqlite/ packages/tar/ packages/termcap/ packages/terminal/ packages/viz/ packages/wasi-js/ packages/wasm-opt/ packages/zlib/

TARGET=$1
shift
ROOTDIR=`pwd`

for PACKAGE in $*
do
    cd "$ROOTDIR/$PACKAGE"
    echo $PACKAGE
    if [ $TARGET = "all" ]; then
        make
    else
        make $TARGET
    fi
done
packages/browser/
make[1]: Entering directory '/home/ryutas/packages/cowasm/cowasm/packages/browser'
npm ci

added 471 packages, and audited 1886 packages in 9s

102 packages are looking for funding
  run `npm fund` for details

1 high severity vulnerability

To address all issues, run:
  npm audit fix

Run `npm audit` for details.
npm run build

> @cowasm/browser@0.3.0 build
> webpack

asset main.bundle.js 461 KiB [emitted] (name: main)
asset index.html 246 bytes [emitted]
runtime modules 1.13 KiB 5 modules
modules by path ./node_modules/ 156 KiB
  modules by path ./node_modules/assert/build/ 67.8 KiB 4 modules
  modules by path ./node_modules/object-is/*.js 1.14 KiB 4 modules
  modules by path ./node_modules/is-nan/*.js 1.2 KiB 4 modules
  modules by path ./node_modules/util/ 27.9 KiB 3 modules
  modules by path ./node_modules/object-keys/*.js 4.36 KiB 3 modules
  modules by path ./node_modules/debug/src/*.js 12 KiB 2 modules
  modules by path ./node_modules/call-bind/*.js 1.68 KiB 2 modules
  modules by path ./node_modules/function-bind/*.js 1.55 KiB 2 modules
  modules by path ./node_modules/has-symbols/*.js 2.13 KiB 2 modules
  + 18 modules
modules by path ./src/*.ts 9.31 KiB
  ./src/index.ts 2.94 KiB [built] [code generated]
  ./src/python.ts 6.37 KiB [built] [code generated] [1 error]

LOG from webpack.FileSystemInfo
<w> Managed item /home/ryutas/packages/cowasm/cowasm/packages/browser/node_modules/python-wasm isn't a directory or doesn't contain a package.json (see snapshot.managedPaths option)
+ 12 hidden lines

ERROR in ./src/python.ts 38:0-37
Module not found: Error: Can't resolve 'python-wasm' in '/home/ryutas/packages/cowasm/cowasm/packages/browser/src'
 @ ./src/index.ts 39:0-30 46:37-43

ERROR in /home/ryutas/packages/cowasm/cowasm/packages/browser/src/kernel.ts
3:28-44
[tsl] ERROR in /home/ryutas/packages/cowasm/cowasm/packages/browser/src/kernel.ts(3,29)
      TS2306: File '/home/ryutas/packages/cowasm/cowasm/packages/kernel/index.d.ts' is not a module.

ERROR in /home/ryutas/packages/cowasm/cowasm/packages/browser/src/python.ts
./src/python.ts 3:23-36
[tsl] ERROR in /home/ryutas/packages/cowasm/cowasm/packages/browser/src/python.ts(3,24)
      TS2307: Cannot find module 'python-wasm' or its corresponding type declarations.
 @ ./src/index.ts 39:0-30 46:37-43

3 errors have detailed information that is not shown.
Use 'stats.errorDetails: true' resp. '--stats-error-details' to show it.

webpack 5.73.0 compiled with 3 errors in 1531 ms
make[1]: *** [Makefile:9: /home/ryutas/packages/cowasm/cowasm/packages/browser/dist/.built] Error 1
make[1]: Leaving directory '/home/ryutas/packages/cowasm/cowasm/packages/browser'
make: *** [Makefile:10: all] Error 2
williamstein commented 1 year ago

After the latest refactor I got this to work, building everything (including all the python packages like pandas and numpy too) all on archlinux. There were numerous subtle little issues with make vars, etc., that I had to fix. Also I figured out that this is the minimal list of packages to install:

pacman -Sy binutils git nodejs npm cmake curl m4 yasm texinfo python libtool tcl zip unzip patch binutils diffutils

I did my build testing in a Docker container on x86_64 Linux, started using

docker run -it archlinux bash

Then just install the above, git clone https://github.com/sagemathinc/cowasm/, cd in and make.

williamstein commented 1 year ago

Oh, the full test suite (via make test) also passed on aarch linux for me. Yeah!

oroppas commented 1 year ago

Thank you for the fix!

One thing I realized was that pnpm was not part of the standard Arch Linux package. It may be useful to document how to get pnpm. For example, I installed pnpm from https://aur.archlinux.org/packages/pnpm

williamstein commented 1 year ago

Thanks for testing.

It may be useful to document how to get pnpm

Can you make a PR?