janet-lang / jpm

Janet Project Manager
MIT License
68 stars 22 forks source link

compile error: [...] jpm_tree/lib/_jmod_posix_spawn.so: undefined symbol: janet_getmethod #21

Closed Earnestly closed 2 years ago

Earnestly commented 2 years ago

I'm attempting to package both janet and jpm globally while using jpm -l for ultimately building hermes.

When following the installation instructions using jpm -l load-lockfile it appears to build successfully:

% jpm -l load-lockfile
remote: Counting objects: 100% (38/38), done.
remote: Compressing objects: 100% (27/27), done.
remote: Total 95 (delta 16), reused 23 (delta 10), pack-reused 57
Receiving objects: 100% (95/95), 4.23 MiB | 4.23 MiB/s, done.
Resolving deltas: 100% (45/45), done.
./project.janet:12:10: compile warning (normal): file/popen is deprecated
compiling sqlite3.c to build/sqlite3.o...
compiling main.c to build/main.o...
compiling sqlite3.c to build/sqlite3.static.o...
compiling main.c to build/main.static.o...
generating meta file build/sqlite3.meta.janet...
linking build/sqlite3.so...
creating static library build/sqlite3.a...
generating /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.manifests/sqlite3.jdn...
Installed as 'sqlite3'.
copying build/sqlite3.so to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
copying build/sqlite3.meta.janet to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
copying build/sqlite3.a to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
cloning repository https://github.com/janet-lang/argparse.git to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_30acacff4f3b1466d92440264f93dbdd2e8ab5ed_https___github.com_janet-lang_argparse.git
Cloning into '/home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_30acacff4f3b1466d92440264f93dbdd2e8ab5ed_https___github.com_janet-lang_argparse.git'...
remote: Enumerating objects: 120, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (15/15), done.
remote: Total 120 (delta 3), reused 7 (delta 2), pack-reused 101
Receiving objects: 100% (120/120), 26.06 KiB | 13.03 MiB/s, done.
Resolving deltas: 100% (51/51), done.
generating /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.manifests/argparse.jdn...
Installed as 'argparse'.
copying argparse.janet to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
cloning repository https://github.com/janet-lang/path.git to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_0ae7b60b8aaaa7f80f84692b7efb8e46b7d38eb3_https___github.com_janet-lang_path.git
Cloning into '/home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_0ae7b60b8aaaa7f80f84692b7efb8e46b7d38eb3_https___github.com_janet-lang_path.git'...
remote: Enumerating objects: 52, done.
remote: Total 52 (delta 0), reused 0 (delta 0), pack-reused 52
Receiving objects: 100% (52/52), 9.25 KiB | 4.63 MiB/s, done.
Resolving deltas: 100% (27/27), done.
generating /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.manifests/path.jdn...
Installed as 'path'.
copying path.janet to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
cloning repository https://github.com/andrewchambers/janet-uri.git to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_a080bde1d9d46cf3310190d5a1143129a0b07f10_https___github.com_andrewchambers_janet-uri.git
Cloning into '/home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_a080bde1d9d46cf3310190d5a1143129a0b07f10_https___github.com_andrewchambers_janet-uri.git'...
remote: Enumerating objects: 79, done.
remote: Counting objects: 100% (79/79), done.
remote: Compressing objects: 100% (50/50), done.
remote: Total 79 (delta 36), reused 56 (delta 22), pack-reused 0
Receiving objects: 100% (79/79), 16.37 KiB | 364.00 KiB/s, done.
Resolving deltas: 100% (36/36), done.
compiling uri.c to build/uri.o...
compiling uri.c to build/uri.static.o...
generating meta file build/_uri.meta.janet...
creating static library build/_uri.a...
linking build/_uri.so...
generating /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.manifests/uri.jdn...
Installed as 'uri'.
copying uri.janet to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
copying build/_uri.so to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
copying build/_uri.meta.janet to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
copying build/_uri.a to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
cloning repository https://github.com/andrewchambers/janet-jdn.git to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_2268b81004bb9b58e8b263584c86219dab8180ac_https___github.com_andrewchambers_janet-jdn.git
Cloning into '/home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_2268b81004bb9b58e8b263584c86219dab8180ac_https___github.com_andrewchambers_janet-jdn.git'...
remote: Enumerating objects: 29, done.
remote: Counting objects: 100% (29/29), done.
remote: Compressing objects: 100% (21/21), done.
remote: Total 29 (delta 10), reused 19 (delta 6), pack-reused 0
Receiving objects: 100% (29/29), 5.45 KiB | 5.45 MiB/s, done.
Resolving deltas: 100% (10/10), done.
generating /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.manifests/jdn.jdn...
Installed as 'jdn'.
copying jdn.janet to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
cloning repository https://github.com/andrewchambers/janet-flock.git to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_3ec2f6e3345f347dc9d22b0aeea2e1b445ae8027_https___github.com_andrewchambers_janet-flock.git
Cloning into '/home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_3ec2f6e3345f347dc9d22b0aeea2e1b445ae8027_https___github.com_andrewchambers_janet-flock.git'...
remote: Enumerating objects: 59, done.
remote: Counting objects: 100% (59/59), done.
remote: Compressing objects: 100% (38/38), done.
remote: Total 59 (delta 29), reused 47 (delta 17), pack-reused 0
Receiving objects: 100% (59/59), 7.63 KiB | 7.63 MiB/s, done.
Resolving deltas: 100% (29/29), done.
compiling flock.c to build/flock.o...
generating meta file build/flock.meta.janet...
compiling flock.c to build/flock.static.o...
creating static library build/flock.a...
linking build/flock.so...
generating /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.manifests/flock.jdn...
Installed as 'flock'.
copying build/flock.so to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
copying build/flock.meta.janet to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
copying build/flock.a to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
cloning repository https://github.com/andrewchambers/janet-fork.git to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_39c887955781679915bbd812b6fc65cf053fd5d9_https___github.com_andrewchambers_janet-fork.git
Cloning into '/home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_39c887955781679915bbd812b6fc65cf053fd5d9_https___github.com_andrewchambers_janet-fork.git'...
remote: Enumerating objects: 18, done.
remote: Counting objects: 100% (18/18), done.
remote: Compressing objects: 100% (15/15), done.
remote: Total 18 (delta 4), reused 14 (delta 2), pack-reused 0
Receiving objects: 100% (18/18), 4.31 KiB | 4.31 MiB/s, done.
Resolving deltas: 100% (4/4), done.
compiling fork.c to build/fork.static.o...
compiling fork.c to build/fork.o...
generating meta file build/_jmod_fork.meta.janet...
creating static library build/_jmod_fork.a...
linking build/_jmod_fork.so...
generating /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.manifests/fork.jdn...
Installed as 'fork'.
copying build/_jmod_fork.so to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
copying build/_jmod_fork.meta.janet to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
copying build/_jmod_fork.a to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
copying fork.janet to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
cloning repository https://github.com/andrewchambers/janet-base16.git to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_7e740b612e3f65ea5a8655e99f28926128a6b292_https___github.com_andrewchambers_janet-base16.git
Cloning into '/home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_7e740b612e3f65ea5a8655e99f28926128a6b292_https___github.com_andrewchambers_janet-base16.git'...
remote: Enumerating objects: 17, done.
remote: Counting objects: 100% (17/17), done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 17 (delta 4), reused 5 (delta 0), pack-reused 0
Receiving objects: 100% (17/17), 4.69 KiB | 4.69 MiB/s, done.
Resolving deltas: 100% (4/4), done.
compiling base16.c to build/base16.o...
generating meta file build/base16.meta.janet...
compiling base16.c to build/base16.static.o...
creating static library build/base16.a...
linking build/base16.so...
generating /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.manifests/base16.jdn...
Installed as 'base16'.
copying build/base16.so to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
copying build/base16.meta.janet to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
copying build/base16.a to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
cloning repository https://github.com/andrewchambers/janet-shlex.git to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_f9762ce5b360a855f0b05cfcf1c93d4427b43bbd_https___github.com_andrewchambers_janet-shlex.git
Cloning into '/home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_f9762ce5b360a855f0b05cfcf1c93d4427b43bbd_https___github.com_andrewchambers_janet-shlex.git'...
remote: Enumerating objects: 21, done.
remote: Counting objects: 100% (21/21), done.
remote: Compressing objects: 100% (15/15), done.
remote: Total 21 (delta 7), reused 16 (delta 5), pack-reused 0
Receiving objects: 100% (21/21), done.
Resolving deltas: 100% (7/7), done.
generating /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.manifests/shlex.jdn...
Installed as 'shlex'.
copying shlex.janet to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
cloning repository https://github.com/andrewchambers/janet-posix-spawn.git to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_adbc63a773765481e7117752ba75db548667101d_https___github.com_andrewchambers_janet-posix-spawn.git
Cloning into '/home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_adbc63a773765481e7117752ba75db548667101d_https___github.com_andrewchambers_janet-posix-spawn.git'...
remote: Enumerating objects: 60, done.
remote: Counting objects: 100% (60/60), done.
remote: Compressing objects: 100% (43/43), done.
remote: Total 60 (delta 31), reused 42 (delta 16), pack-reused 0
Receiving objects: 100% (60/60), 14.36 KiB | 980.00 KiB/s, done.
Resolving deltas: 100% (31/31), done.
compiling posix-spawn.c to build/posix-spawn.o...
compiling posix-spawn.c to build/posix-spawn.static.o...
generating meta file build/_jmod_posix_spawn.meta.janet...
linking build/_jmod_posix_spawn.so...
creating static library build/_jmod_posix_spawn.a...
generating /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.manifests/posix-spawn.jdn...
Installed as 'posix-spawn'.
copying build/_jmod_posix_spawn.so to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
copying build/_jmod_posix_spawn.meta.janet to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
copying build/_jmod_posix_spawn.a to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
copying posix-spawn.janet to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
cloning repository https://github.com/andrewchambers/janet-sh.git to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_4226a4cae548fde38358aa1fdb6d6a90bf5fe175_https___github.com_andrewchambers_janet-sh.git
Cloning into '/home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.cache/git_4226a4cae548fde38358aa1fdb6d6a90bf5fe175_https___github.com_andrewchambers_janet-sh.git'...
remote: Enumerating objects: 94, done.
remote: Counting objects: 100% (94/94), done.
remote: Compressing objects: 100% (57/57), done.
remote: Total 94 (delta 38), reused 85 (delta 32), pack-reused 0
Receiving objects: 100% (94/94), 20.84 KiB | 2.08 MiB/s, done.
Resolving deltas: 100% (38/38), done.
compiling _sh.c to build/_sh.static.o...
compiling _sh.c to build/_sh.o...
generating meta file build/_sh.meta.janet...
creating static library build/_sh.a...
linking build/_sh.so...
generating /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/.manifests/sh.jdn...
Installed as 'sh'.
copying build/_sh.so to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
copying build/_sh.meta.janet to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
copying build/_sh.a to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...
copying sh.janet to /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib...

However then using jpm -l build I receive the following error:

% jpm -l build
/usr/lib/janet/jpm/shutil.janet:68:25: compile warning (normal): file/popen is deprecated
./project.janet:1:1: compile error: error: (macro) could not load native /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/_jmod_posix_spawn.so: /home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/_jmod_posix_spawn.so: undefined symbol: janet_getmethod
  in native [src/core/corelib.c] on line 314
  in native-loader [boot.janet] on line 2660, column 40
  in require-1 [boot.janet] on line 2683, column 18
  in import* [boot.janet] on line 2714, column 15
  in _thunk [/home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/posix-spawn.janet] (tailcall) on line 1, column 1
  in dofile [boot.janet] (tailcall) on line 2652, column 7
  in source-loader [boot.janet] on line 2663, column 15
  in require-1 [boot.janet] on line 2683, column 18
  in import* [boot.janet] on line 2714, column 15
  in _thunk [/home/earnest/hermes/hermes-git/src/hermes/jpm_tree/lib/sh.janet] (tailcall) on line 1, column 1
  in dofile [boot.janet] (tailcall) on line 2652, column 7
  in source-loader [boot.janet] on line 2663, column 15
  in require-1 [boot.janet] on line 2683, column 18
  in import* [boot.janet] on line 2714, column 15
  in _thunk [eval] (tailcall) on line 3, column 3
  in reduce [boot.janet] on line 834, column 27
  in post-deps [/usr/lib/janet/jpm/pm.janet] on line 252, column 8

Could you provide any guidance on what I may have done wrong or how to investigate what may be causing this linking failure?

I'm not sure what information to provide but here is some which may be helpful:

% janet -v
1.18.0-dev-ba3107c1 # (built with `-fPIC` for full RELRO)

% jpm show-paths
/usr/lib/janet/jpm/shutil.janet:68:25: compile warning (normal): file/popen is deprecated
binpath:    /usr/bin
modpath:    /usr/lib/janet
libpath:    /usr/lib
headerpath: /usr/include/janet
syspath:    /usr/lib/janet
andrewchambers commented 2 years ago

Hi there, just a note, I am the author of hermes, and have continued development of a new package tree based on the lessons learned from hermes that I will be releasing later.

That being said, what is there should still work, though jpm has been through a rewrite in the meantime. I am not totally sure if this is a hermes problem, or a problem with the new jpm, or with the packaging of janet.

Earnestly commented 2 years ago

That is what I hope to discover as I'm not sure where exactly the problem lies.

bakpakin commented 2 years ago

Can you build any native executables? One of the simplest is https://github.com/janet-lang/json. Just a wild guess, but you sometimes see an error like that if janet is compiled without rdynamic. You would only see that though if you are really messing with meson or the Makefile, or just bypassing them altogether.

Earnestly commented 2 years ago

Good call, it does appear to render the same type of failure:

% git clone https://github.com/janet-lang/json
Cloning into 'json'...
remote: Enumerating objects: 65, done.
remote: Counting objects: 100% (4/4), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 65 (delta 0), reused 2 (delta 0), pack-reused 61
Receiving objects: 100% (65/65), 16.48 KiB | 937.00 KiB/s, done.
Resolving deltas: 100% (33/33), done.

% cd json
% ~/json jpm -l build
/usr/lib/janet/jpm/shutil.janet:68:25: compile warning (normal): file/popen is deprecated
generating meta file build/json.meta.janet...
compiling json.c to build/json.o...
compiling json.c to build/json.static.o...
linking build/json.so...
creating static library build/json.a...

% ~/json jpm -l test
/usr/lib/janet/jpm/shutil.janet:68:25: compile warning (normal): file/popen is deprecated
running test/suite0.janet ...
could not load native build/json.so: build/json.so: undefined symbol: janet_optbuffer
  in native [src/core/corelib.c] on line 314
  in native-loader [boot.janet] on line 2660, column 40
  in require-1 [boot.janet] on line 2683, column 18
  in import* [boot.janet] on line 2714, column 15
  in _thunk [test/suite0.janet] (tailcall) on line 1, column 1
error: non-zero exit code in test/suite0.janet: 1
  in errorf [boot.janet] on line 165, column 3
  in dodir [/usr/lib/janet/jpm/declare.janet] on line 277, column 19
  in run-tests [/usr/lib/janet/jpm/declare.janet] (tailcall) on line 279, column 3
  in <anonymous> [/usr/lib/janet/jpm/rules.janet] on line 18, column 20
  in executor [/usr/lib/janet/jpm/rules.janet] on line 25, column 8
  in worker [/usr/lib/janet/jpm/dagbuild.janet] on line 65, column 23
  in <anonymous> [/usr/lib/janet/jpm/dagbuild.janet] on line 21, column 43
  in pmap [/usr/lib/janet/jpm/dagbuild.janet] on line 26, column 7
  in pdag [/usr/lib/janet/jpm/dagbuild.janet] (tailcall) on line 71, column 3
  in _thunk [/usr/bin/jpm] (tailcall) on line -1, column -1
  in cli-main [boot.janet] on line 3644, column 17

You mentioned -rdynamic but I thought the Makefile covered this with: LDFLAGS?=-rdynamic. However I've since checked the compiler output again and fail to see -rdynamic during linking:

cc -DJANET_BOOTSTRAP -DJANET_BUILD="\"038ca1b9\"" -O0 -g -std=c99 -Wall -Wextra -Isrc/include -Isrc/conf -fvisibility=hidden -fPIC -o [...] -lm -lpthread -lrt -ldl
build/janet_boot . JANET_PATH '/usr/lib/janet' > build/c/janet.c
cksum build/c/janet.c
3129288624 2240543 build/c/janet.c
cc -O2 -std=c99 -Wall -Wextra -Isrc/include -Isrc/conf -fvisibility=hidden -fPIC -c build/c/janet.c -o build/janet.o
cp src/mainclient/shell.c build/c/shell.c
cc -O2 -std=c99 -Wall -Wextra -Isrc/include -Isrc/conf -fvisibility=hidden -fPIC -c build/c/shell.c -o build/shell.o
cc -rdynamic -O2 -std=c99 -Wall -Wextra -Isrc/include -Isrc/conf -fvisibility=hidden -fPIC -o build/janet build/janet.o build/shell.o -lm -lpthread -lrt -ldl
cc -rdynamic -O2 -std=c99 -Wall -Wextra -Isrc/include -Isrc/conf -fvisibility=hidden -fPIC -Wl,-soname,libjanet.so.1.18 -shared -o build/libjanet.so build/janet.o build/shell.o -lm -lpthread -lrt -ldl
ar rcs build/libjanet.a build/janet.o build/shell.o

Further it also seems to ignore the environmental flags used by package management:

CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection"
CXXFLAGS="$CFLAGS -Wp,-D_GLIBCXX_ASSERTIONS"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now"

This may need much more work than I was expecting.

Earnestly commented 2 years ago

I've since switched the build system to meson and it appears to work; it does use -rdynamic when linking.

Although my prefered CFLAGS are still not honoured but this can be sorted out later.

I'm not sure if you wish to close this issue as there does appear to be a divergence when using the make build system but otherwise it appears resolved.

bakpakin commented 2 years ago

My bad, -rdynamic should be part of LDFLAGS, that should fix the issue. Anyway, using meson is also a good solution and that is equally supported.