search

gambit / gambit

Gambit is an efficient implementation of the Scheme programming language.
http://gambitscheme.org/
1.33k stars 168 forks source link

Gambit building with smp has been broken since fbd34d2e. #912

Open ober opened 4 months ago

ober commented 4 months ago

fbd34d2e * Use eager gc_hash_table rehashing when --enable-multiple-threaded-vms

ober commented 4 months ago

built with ./configure CC=gcc --enable-multiple-threaded-vms --enable-smp --enable-debug --enable-debug-log --enable-debug-c-backtrace --enable-debug-ctrl-flow-history --enable-default-runtime-options=f8,-8,t8 --enable-dynamic-clib --enable-march=native --enable-poll --enable-openssl --enable-single-host --prefix=/opt/gambit

afleckenstein commented 4 months ago

I can reproduce this,

./configure --enable-single-host --enable-targets= --enable-multiple-threaded-vms --enable-smp --enable-debug-garbage-collect make bootstrap; make bootclean succeeds, the make afterward fails on one of my machines with:

../gsc-boot -:~~bin=../bin,~~lib=../lib,~~include=../include -f -c  -prelude "(define-cond-expand-feature|enable-type-checking|)(define-cond-expand-feature|disable-auto-forcing|)(define-cond-expand-feature|enable-sharp-dot|)(define-cond-expand-feature|enable-bignum|)(define-cond-expand-feature|enable-ratnum|)(define-cond-expand-feature|enable-cpxnum|)(define-cond-expand-feature|enable-s8vector|)(define-cond-expand-feature|enable-u16vector|)(define-cond-expand-feature|enable-s16vector|)(define-cond-expand-feature|enable-u32vector|)(define-cond-expand-feature|enable-s32vector|)(define-cond-expand-feature|enable-u64vector|)(define-cond-expand-feature|enable-s64vector|)(define-cond-expand-feature|enable-f32vector|)(define-cond-expand-feature|enable-smp|)(##include\"../lib/header.scm\")" _io.scm
*** ERROR -- Compiler internal error detected
*** in procedure targ-get-prim-info, unknown primitive: "##not"
*** ERROR IN c#targ-setup-inlinable -- Internal error, no exception handler at this point #f
make[1]: *** [makefile:183: _io.c] Error 70
make[1]: Leaving directory '/home/andy/src/gambit/lib'
make: *** [makefile:544: core-recursive] Error 1

i have also (on a different machine) just gotten a segfault while building _io.scm.

I would be curious to know how to make use of the enable-debug-garbage-collect, or just more info on how to debug a compiler internal error. the above error is confusing to me, my intuition is that the "unknown ##not" is somewhat of a secondary issue, something like an error while trying to report an error.

gambiteer commented 4 months ago

I see a failure with the previous checkin, too:

    86  15:53   git checkout 0e034f1
    87  15:53   make clean
    88  15:54   /bin/rm -rf boot gsc-boot
    89  15:55   ./configure CC=gcc-14 --enable-single-host --enable-targets= --enable-multiple-threaded-vms --enable-smp --enable-debug-garbage-collect
    90  15:55   make -j8
<stuff deleted>
*** to compile the builtin modules you should: make modules
/Library/Developer/CommandLineTools/usr/bin/make modules  # comment this line during development to avoid building the modules on every change
>>>> building module _define-library for C
*** FATAL ERROR -- GC inconsistency detected
>>>> building module _define-library/define-library-expand for C
*** FATAL ERROR -- GC inconsistency detected
>>>> building module _define-library/debug for C
*** FATAL ERROR -- GC inconsistency detected
make[3]: *** [modules-pre] Error 70
make[2]: *** [_define-library] Error 1
make[1]: *** [modules] Error 2
make: *** [all] Error 2

Then to retry

[MacBook-Pro:~/programs/gambit/gambit] lucier% make
/Library/Developer/CommandLineTools/usr/bin/make stamp
#define ___STAMP_VERSION "v4.9.5-121-g0e034f10"
___STAMP_VERSION unchanged from previous build
making core in include
making core in bin
making core in lib
/Library/Developer/CommandLineTools/usr/bin/make deselect-gen-for-commit-post-nonrec
if [ -e ../.git ]; then git update-index --skip-worktree _kernel.c _system.c _num.c _std.c _eval.c _module.c _io.c _nonstd.c _thread.c _repl.c _gambit.c; fi
making core in gsi
/Libxrary/Developer/CommandLineTools/usr/bin/make deselect-gen-for-commit-post-nonrec
if [ -e ../.git ]; then git update-index --skip-worktree _gsilib.c _gsi.c _gambitgsi.c _gsi_.c; fi
making core in gsc
/Library/Developer/CommandLineTools/usr/bin/make deselect-gen-for-commit-post-nonrec
if [ -e ../.git ]; then git update-index --skip-worktree _host.c _utils.c _source.c _parms.c _env.c _ptree1.c _ptree2.c _gvm.c _back.c _front.c _prims.c _assert.c _asm.c _x86.c _arm.c _riscv.c _codegen.c _t-univ-1.c _t-univ-2.c _t-univ-3.c _t-univ-4.c _t-cpu-abstract-machine.c _t-cpu-primitives.c _t-cpu-object-desc.c _t-cpu-utils.c _t-cpu-backend-x86.c _t-cpu-backend-arm.c _t-cpu-backend-riscv.c _t-cpu.c _t-c-1.c _t-c-2.c _t-c-3.c _gsclib.c _gsc.c _gambitgsc.c _gsc_.c; fi
making core in misc
making core in doc
making core in tests
making core in examples
making core in distr-comp
making core in pi
making core in ring
making core in web-repl
making core in web-server
making core in tcltk
making core in Xlib-simple
making core in pthread
making core in misc
making core in contrib
making core in GambitREPL
making core in try
making core in xactlog
making core in prebuilt
making core in macosx
making core in windows
making core in bench
making core in githooks
*** to compile the builtin modules you should: make modules
/Library/Developer/CommandLineTools/usr/bin/make modules  # comment this line during development to avoid building the modules on every change
>>>> building module _define-library for C
*** FATAL ERROR -- GC inconsistency detected
>>>> no need to build module _define-library/define-library-expand for C
>>>> no need to build module _define-library/debug for C
>>>> building module _base64 for C
*** FATAL ERROR -- GC inconsistency detected
make[4]: *** [modules-pre] Error 70
make[3]: *** [_base64] Error 1
make[2]: *** [modules-recursive] Error 2
make[1]: *** [modules] Error 2
make: *** [all] Error 2
[MacBook-Pro:~/programs/gambit/gambit] lucier% git branch
* (HEAD detached at 0e034f10)
  master
[MacBook-Pro:~/programs/gambit/gambit] lucier% git log
commit 0e034f100104d981eca1e88049ba753f83d284cc (HEAD)
Author: Bradley Lucier <lucier@math.purdue.edu>
Date:   Thu Feb 15 14:01:28 2024 -0500

    Mimic https://github.com/gambit/gambit/commit/e25a37038106d87416419130f41654f42742651f to fix mismatched vector arg lengths

This is on M1 MacBook Pro with Homebrew gcc-14. I'm including gambit.log, which has some information about the inconsistency. gambit.log

So do you see anything like this with this as the last commit?

gambiteer commented 4 months ago

I also get a similar failure with https://github.com/gambit/gambit/commit/fbd34d2e4af139ed5d9621ef1f279382e2add813:

   107  16:43   git checkout fbd34d2
   108  16:43   make clean
   109  16:43   /bin/rm -rf boot gsc-boot
   110  16:44   ./configure CC=gcc-14 --enable-single-host --enable-targets= --enable-multiple-threaded-vms --enable-smp --enable-debug-garbage-collect
   111  16:44   make -k8
   112  16:44   make -j8
<stuff deleted>
*** to compile the builtin modules you should: make modules
/Library/Developer/CommandLineTools/usr/bin/make modules  # comment this line during development to avoid building the modules on every change
>>>> building module _define-library for C
/bin/sh: line 1: 36850 Segmentation fault: 11  ../../gsc/gsc -:~~bin=../../bin,~~lib=../../lib,~~include=../../include -f -e "(let ((x (##build-module \"$mod_dir$main_mod\" (quote $target) (quote ((module-ref $modref)(warnings)))))) (if (not x) (begin (println \"*** module failed to build\") (exit 1))))"
>>>> building module _define-library/define-library-expand for C
/bin/sh: line 1: 36851 Segmentation fault: 11  ../../gsc/gsc -:~~bin=../../bin,~~lib=../../lib,~~include=../../include -f -e "(let ((x (##build-module \"$mod_dir$main_mod\" (quote $target) (quote ((module-ref $modref)(warnings)))))) (if (not x) (begin (println \"*** module failed to build\") (exit 1))))"
>>>> building module _define-library/debug for C
/bin/sh: line 1: 36852 Segmentation fault: 11  ../../gsc/gsc -:~~bin=../../bin,~~lib=../../lib,~~include=../../include -f -e "(let ((x (##build-module \"$mod_dir$main_mod\" (quote $target) (quote ((module-ref $modref)(warnings)))))) (if (not x) (begin (println \"*** module failed to build\") (exit 1))))"
make[3]: *** [modules-pre] Error 139
make[2]: *** [_define-library] Error 1
make[1]: *** [modules] Error 2
make: *** [all] Error 2

Restart build with serial make:

[MacBook-Pro:~/programs/gambit/gambit] lucier% make
/Library/Developer/CommandLineTools/usr/bin/make stamp
#define ___STAMP_VERSION "v4.9.5-122-gfbd34d2e"
___STAMP_VERSION unchanged from previous build
making core in include
making core in bin
making core in lib
/Library/Developer/CommandLineTools/usr/bin/make deselect-gen-for-commit-post-nonrec
if [ -e ../.git ]; then git update-index --skip-worktree _kernel.c _system.c _num.c _std.c _eval.c _module.c _io.c _nonstd.c _thread.c _repl.c _gambit.c; fi
making core in gsi
/Library/Developer/CommandLineTools/usr/bin/make deselect-gen-for-commit-post-nonrec
if [ -e ../.git ]; then git update-index --skip-worktree _gsilib.c _gsi.c _gambitgsi.c _gsi_.c; fi
making core in gsc
/Library/Developer/CommandLineTools/usr/bin/make deselect-gen-for-commit-post-nonrec
if [ -e ../.git ]; then git update-index --skip-worktree _host.c _utils.c _source.c _parms.c _env.c _ptree1.c _ptree2.c _gvm.c _back.c _front.c _prims.c _assert.c _asm.c _x86.c _arm.c _riscv.c _codegen.c _t-univ-1.c _t-univ-2.c _t-univ-3.c _t-univ-4.c _t-cpu-abstract-machine.c _t-cpu-primitives.c _t-cpu-object-desc.c _t-cpu-utils.c _t-cpu-backend-x86.c _t-cpu-backend-arm.c _t-cpu-backend-riscv.c _t-cpu.c _t-c-1.c _t-c-2.c _t-c-3.c _gsclib.c _gsc.c _gambitgsc.c _gsc_.c; fi
making core in misc
making core in doc
making core in tests
making core in examples
making core in distr-comp
making core in pi
making core in ring
making core in web-repl
making core in web-server
making core in tcltk
making core in Xlib-simple
making core in pthread
making core in misc
making core in contrib
making core in GambitREPL
making core in try
making core in xactlog
making core in prebuilt
making core in macosx
making core in windows
making core in bench
making core in githooks
*** to compile the builtin modules you should: make modules
/Library/Developer/CommandLineTools/usr/bin/make modules  # comment this line during development to avoid building the modules on every change
>>>> building module _define-library for C
*** FATAL ERROR -- GC inconsistency detected
>>>> building module _define-library/define-library-expand for C
*** FATAL ERROR -- GC inconsistency detected
>>>> building module _define-library/debug for C
/bin/sh: line 1: 37093 Segmentation fault: 11  ../../gsc/gsc -:~~bin=../../bin,~~lib=../../lib,~~include=../../include -f -e "(let ((x (##build-module \"$mod_dir$main_mod\" (quote $target) (quote ((module-ref $modref)(warnings)))))) (if (not x) (begin (println \"*** module failed to build\") (exit 1))))"
make[3]: *** [modules-pre] Error 139
make[2]: *** [_define-library] Error 1
make[1]: *** [modules] Error 2
make: *** [all] Error 2

For some reason I don't find a new gambit.log file that describes the inconsistencys.

afleckenstein commented 4 months ago

@gambiteer I can build 0e034f1 just fine. I never see 'GC inconsistency detected', instead I get errors about missing generated C source files

*** START OF DEBUGGING TRACES
fd=0 kind=159 direction=1
fd=1 kind=159 direction=2
fd=2 kind=159 direction=2
********* used fudge: stack = 0  heap = 1
********* used fudge: stack = 0  heap = 3
********* used fudge: stack = 0  heap = 2
********* used fudge: stack = 0  heap = 1
path="/home/andy/src/gambitsmp/gambit/boot/gsc/../lib/gambext" fl=2048
*** OS ERROR AT "os_io.c"@9563.1 -- errno=2 (No such file or directory)
********* used fudge: stack = 0  heap = 14
*** OS ERROR AT "os_setup.c"@2177.1 -- errno=2 (No such file or directory)
*** OS ERROR AT "os_setup.c"@2177.1 -- errno=2 (No such file or directory)
*** OS ERROR AT "os_setup.c"@2177.1 -- errno=2 (No such file or directory)
*** OS ERROR AT "os_setup.c"@2177.1 -- errno=2 (No such file or directory)
*** OS ERROR AT "os_files.c"@1542.1 -- errno=20 (Not a directory)
path="/home/andy/src/gambitsmp/gambit/boot/gsc/_gsc.c" fl=2048
fd=4 kind=63 direction=1
file char_encoding_errors=32   char_encoding=3   eol_encoding=128   buffering=1536
*** OS ERROR AT "os_files.c"@1542.1 -- errno=20 (Not a directory)
path="/home/andy/src/gambitsmp/gambit/boot/gsc/_gsc.c" fl=2048
fd=4 kind=63 direction=1
file char_encoding_errors=32   char_encoding=3   eol_encoding=128   buffering=1536
path="/home/andy/src/gambitsmp/gambit/boot/gsc/_gambitgsc.c" fl=2048
fd=4 kind=63 direction=1
file char_encoding_errors=32   char_encoding=3   eol_encoding=128   buffering=1536
********* used fudge: stack = 0  heap = 4
********* used fudge: stack = 0  heap = 5
********* used fudge: stack = 0  heap = 1
********* used fudge: stack = 0  heap = 8
********* used fudge: stack = 0  heap = 2
********* used fudge: stack = 0  heap = 2
********* used fudge: stack = 0  heap = 3

which is puzzling, and also later on there is negative heap usage:

----------------------------------------- GC START #1
processor #0
heap_size          = 1703936
tospace_offset     = 0
___ps->stack_start = 0x7f2c27f9a030
___ps->stack_break = 0x7f2c27f99cd0
___ps->fp          = 0x7f2c27f880b0
___ps->stack_limit = 0x7f2c27f69f90
___ps->heap_limit  = 0x7f2c27422e30
___ps->hp          = 0x7f2c27422e48
garbage_collect_setup_phase
garbage_collect_mark_strong_phase
garbage_collect_mark_weak_phase
garbage_collect_cleanup_phase
----------------------------------------- GC END #1
********* used fudge: stack = 0  heap = 2
********* used fudge: stack = 0  heap = 1
********* used fudge: stack = 14  heap = -3097
********* used fudge: stack = 0  heap = 2
********* used fudge: stack = 14  heap = -32119
********* used fudge: stack = 14  heap = -2989
********* used fudge: stack = 0  heap = 2
********* used fudge: stack = 14  heap = -30223
********* used fudge: stack = 14  heap = -4314
********* used fudge: stack = 0  heap = 1
********* used fudge: stack = 14  heap = -16637
********* used fudge: stack = 0  heap = 2
********* used fudge: stack = 14  heap = -12146
********* used fudge: stack = 0  heap = 2
path="/home/andy/src/gambitsmp/gambit/boot/gsc/_gsc_.c" fl=2625
fd=4 kind=63 direction=2
file char_encoding_errors=32   char_encoding=3   eol_encoding=128   buffering=1536
********* used fudge: stack = 0  heap = 3
********* used fudge: stack = 0  heap = 4
********* used fudge: stack = 0  heap = 2
********* used fudge: stack = 0  heap = 12
********* used fudge: stack = 0  heap = 7
----------------------------------------- GC START #2

which seems suspicious as well.