asan build failure

[ghost]

git bisect points to https://github.com/jart/cosmopolitan/commit/5b908bc7564397eadd3dda39900b4f32ce9968ed as the first bad commit.

The log around the error:

# ./build/bootstrap/make.com m=asan -j8 o/asan
 1,236,852⏰    772,184⏳  56,520k       8iop o/third_party/gcc/bin/x86_64-linux-musl-g++ -S -Wall -Werror -fdebug-prefix-map=/home/jm/src/cosmopolitan= -frecord-gcc-switches -fno-omit-frame-pointer -fno-optimize-sibling-calls -mno-omit-leaf-frame-pointer -fno-schedule-insns2 -O2 -DSYSDEBUG -fportcosmo -include build/portcosmo.h -DCOSMO -DMODE="asan" -DIMAGE_BASE_VIRTUAL=0x400000 -nostdinc -iquote . -include libc/integral/normalize.inc -msse3 -fno-math-errno -fno-ident -fno-common -fno-gnu-unique -fstrict-aliasing -fstrict-overflow -fno-semantic-interposition -mno-red-zone -mno-tls-direct-seg-refs -fsanitize=address -fno-rtti -fno-exceptions -fuse-cxa-atexit -fno-threadsafe-statics -Wno-int-in-bool-context -Wno-narrowing -Wno-literal-suffix -xc++ -g0 -o /home/jm/src/cosmopolitan/o/tmp/o_asan_third_party_libcxx_string_okk.0176d0/o_asan_third_party_libcxx_string.okk third_party/libcxx/string
third_party/ggml/llama.cc: In function 'size_t llama_set_state_data(llama_context*, const uint8_t*)':
third_party/ggml/llama.cc:2860:1: error: the frame size of 167792 bytes is larger than 131072 bytes [-Werror=frame-larger-than=]
 2860 | }
      | ^
third_party/ggml/llama.cc: In function 'size_t llama_copy_state_data(llama_context*, uint8_t*)':
third_party/ggml/llama.cc:2753:1: error: the frame size of 165328 bytes is larger than 131072 bytes [-Werror=frame-larger-than=]
 2753 | }
      | ^
cc1plus: all warnings being treated as errors

`make MODE=asan -j8 o/asan/third_party/ggml/llama.o` exited with 1:
o/third_party/gcc/bin/x86_64-linux-musl-g++ -msse3 -fno-math-errno -fno-ident -fno-common -fno-gnu-unique -fstrict-aliasing -fstrict-overflow -fno-semantic-interposition -mno-red-zone -mno-tls-direct-seg-refs -fsanitize=address -Wall -Werror -fdebug-prefix-map=/home/jm/src/cosmopolitan= -frecord-gcc-switches -fno-omit-frame-pointer -fno-optimize-sibling-calls -mno-omit-leaf-frame-pointer -fno-schedule-insns2 -O2 -DSYSDEBUG -g -gdwarf-4 -gdescribe-dies -fportcosmo -include build/portcosmo.h -DCOSMO -DMODE="asan" -DIMAGE_BASE_VIRTUAL=0x400000 -nostdinc -iquote . -include libc/integral/normalize.inc -Wa,-W -Wa,-I. -Wa,--noexecstack -Wa,--nocompress-debug-sections -msse3 -fno-math-errno -fno-ident -fno-common -fno-gnu-unique -fstrict-aliasing -fstrict-overflow -fno-semantic-interposition -mno-red-zone -mno-tls-direct-seg-refs -fsanitize=address -fno-rtti -fno-exceptions -fuse-cxa-atexit -fno-threadsafe-statics -Wno-int-in-bool-context -Wno-narrowing -Wno-literal-suffix -c -o /home/jm/src/cosmopolitan/o/tmp/o_asan_third_party_ggml_llama_o.hpd1gu/o_asan_third_party_ggml_llama.o third_party/ggml/llama.cc
consumed 40,145,014µs wall time
ballooned to 399,604kb in size
needed 31,106,141us cpu (8% kernel)
caused 431,149 page faults (100% memcpy)
15,281 context switches (0% consensual)
performed 0 read and 40,488 write i/o operations

make.com: *** [build/rules.mk:32: o/asan/third_party/ggml/llama.o] Error 1
make.com: *** Deleting file 'o/asan/third_party/ggml/llama.o'
make.com: *** Waiting for unfinished jobs....
 1,081,292⏰    626,697⏳  48,704k       8iop o/third_party/gcc/bin/x86_64-linux-musl-g++ -S -Wall -Werror -fdebug-prefix-map=/home/jm/src/cosmopolitan= -frecord-gcc-switches -fno-omit-frame-pointer -fno-optimize-sibling-calls -mno-omit-leaf-frame-pointer -fno-schedule-insns2 -O2 -DSYSDEBUG -fportcosmo -include build/portcosmo.h -DCOSMO -DMODE="asan" -DIMAGE_BASE_VIRTUAL=0x400000 -nostdinc -iquote . -include libc/integral/normalize.inc -msse3 -fno-math-errno -fno-ident -fno-common -fno-gnu-unique -fstrict-aliasing -fstrict-overflow -fno-semantic-interposition -mno-red-zone -mno-tls-direct-seg-refs -fsanitize=address -fno-rtti -fno-exceptions -fuse-cxa-atexit -fno-threadsafe-statics -Wno-int-in-bool-context -Wno-narrowing -Wno-literal-suffix -xc++ -g0 -o /home/jm/src/cosmopolitan/o/tmp/o_asan_third_party_libcxx_string_view_okk.h15upk/o_asan_third_party_libcxx_string_view.okk third_party/libcxx/string_view
   342,335⏰    310,826⏳  32,384k       8iop o/third_party/gcc/bin/x86_64-linux-musl-g++ -S -Wall -Werror -fdebug-prefix-map=/home/jm/src/cosmopolitan= -frecord-gcc-switches -fno-omit-frame-pointer -fno-optimize-sibling-calls -mno-omit-leaf-frame-pointer -fno-schedule-insns2 -O2 -DSYSDEBUG -fportcosmo -include build/portcosmo.h -DCOSMO -DMODE="asan" -DIMAGE_BASE_VIRTUAL=0x400000 -nostdinc -iquote . -include libc/integral/normalize.inc -msse3 -fno-math-errno -fno-ident -fno-common -fno-gnu-unique -fstrict-aliasing -fstrict-overflow -fno-semantic-interposition -mno-red-zone -mno-tls-direct-seg-refs -fsanitize=address -fno-rtti -fno-exceptions -fuse-cxa-atexit -fno-threadsafe-statics -Wno-int-in-bool-context -Wno-narrowing -Wno-literal-suffix -xc++ -g0 -o /home/jm/src/cosmopolitan/o/tmp/o_asan_third_party_libcxx_tuple_okk.nmrrrf/o_asan_third_party_libcxx_tuple.okk third_party/libcxx/tuple
   933,213⏰    772,864⏳  56,924k       8iop o/third_party/gcc/bin/x86_64-linux-musl-g++ -S -Wall -Werror -fdebug-prefix-map=/home/jm/src/cosmopolitan= -frecord-gcc-switches -fno-omit-frame-pointer -fno-optimize-sibling-calls -mno-omit-leaf-frame-pointer -fno-schedule-insns2 -O2 -DSYSDEBUG -fportcosmo -include build/portcosmo.h -DCOSMO -DMODE="asan" -DIMAGE_BASE_VIRTUAL=0x400000 -nostdinc -iquote . -include libc/integral/normalize.inc -msse3 -fno-math-errno -fno-ident -fno-common -fno-gnu-unique -fstrict-aliasing -fstrict-overflow -fno-semantic-interposition -mno-red-zone -mno-tls-direct-seg-refs -fsanitize=address -fno-rtti -fno-exceptions -fuse-cxa-atexit -fno-threadsafe-statics -Wno-int-in-bool-context -Wno-narrowing -Wno-literal-suffix -xc++ -g0 -o /home/jm/src/cosmopolitan/o/tmp/o_asan_third_party_libcxx_system_error_okk.3vocdl/o_asan_third_party_libcxx_system_error.okk third_party/libcxx/system_error
 1,495,244⏰  1,106,280⏳  80,200k       8iop o/third_party/gcc/bin/x86_64-linux-musl-g++ -S -Wall -Werror -fdebug-prefix-map=/home/jm/src/cosmopolitan= -frecord-gcc-switches -fno-omit-frame-pointer -fno-optimize-sibling-calls -mno-omit-leaf-frame-pointer -fno-schedule-insns2 -O2 -DSYSDEBUG -fportcosmo -include build/portcosmo.h -DCOSMO -DMODE="asan" -DIMAGE_BASE_VIRTUAL=0x400000 -nostdinc -iquote . -include libc/integral/normalize.inc -msse3 -fno-math-errno -fno-ident -fno-common -fno-gnu-unique -fstrict-aliasing -fstrict-overflow -fno-semantic-interposition -mno-red-zone -mno-tls-direct-seg-refs -fsanitize=address -fno-rtti -fno-exceptions -fuse-cxa-atexit -fno-threadsafe-statics -Wno-int-in-bool-context -Wno-narrowing -Wno-literal-suffix -xc++ -g0 -o /home/jm/src/cosmopolitan/o/tmp/o_asan_third_party_libcxx_strstream_okk.g5v5oo/o_asan_third_party_libcxx_strstream.okk third_party/libcxx/strstream
 1,243,678⏰    938,830⏳  62,260k       8iop o/third_party/gcc/bin/x86_64-linux-musl-g++ -S -Wall -Werror -fdebug-prefix-map=/home/jm/src/cosmopolitan= -frecord-gcc-switches -fno-omit-frame-pointer -fno-optimize-sibling-calls -mno-omit-leaf-frame-pointer -fno-schedule-insns2 -O2 -DSYSDEBUG -fportcosmo -include build/portcosmo.h -DCOSMO -DMODE="asan" -DIMAGE_BASE_VIRTUAL=0x400000 -nostdinc -iquote . -include libc/integral/normalize.inc -msse3 -fno-math-errno -fno-ident -fno-common -fno-gnu-unique -fstrict-aliasing -fstrict-overflow -fno-semantic-interposition -mno-red-zone -mno-tls-direct-seg-refs -fsanitize=address -fno-rtti -fno-exceptions -fuse-cxa-atexit -fno-threadsafe-statics -Wno-int-in-bool-context -Wno-narrowing -Wno-literal-suffix -xc++ -g0 -o /home/jm/src/cosmopolitan/o/tmp/o_asan_third_party_libcxx_thread_okk.kifoxx/o_asan_third_party_libcxx_thread.okk third_party/libcxx/thread
46,899,340⏰ 35,795,884⏳ 339,968k  54,016iop o/third_party/gcc/bin/x86_64-linux-musl-gcc -msse3 -fno-math-errno -fno-ident -fno-common -fno-gnu-unique -fstrict-aliasing -fstrict-overflow -fno-semantic-interposition -mno-tls-direct-seg-refs -Wall -Werror -fdebug-prefix-map=/home/jm/src/cosmopolitan= -frecord-gcc-switches -fno-optimize-sibling-calls -mno-omit-leaf-frame-pointer -fno-schedule-insns2 -O2 -fno-code-hoisting -DSYSDEBUG -g -gdwarf-4 -gdescribe-dies -fportcosmo -include build/portcosmo.h -DCOSMO -DMODE="asan" -DIMAGE_BASE_VIRTUAL=0x400000 -nostdinc -iquote . -include libc/integral/normalize.inc -Wa,-W -Wa,-I. -Wa,--noexecstack -Wa,--nocompress-debug-sections -msse3 -fno-math-errno -fno-ident -fno-common -fno-gnu-unique -fstrict-aliasing -fstrict-overflow -fno-semantic-interposition -mno-tls-direct-seg-refs -std=gnu2x -O3 -fno-code-hoisting -ffunction-sections -fdata-sections -msse3 -mavx -mavx2 -mf16c -mfma -c -o /home/jm/src/cosmopolitan/o/tmp/o_asan_third_party_ggml_ggml_o.m7d900/o_asan_third_party_ggml_ggml.o third_party/ggml/ggml.c -fdiagnostics-color=always -fsanitize=address -mno-red-zone -D__MNO_RED_ZONE__ -fno-omit-frame-pointer -D__FNO_OMIT_FRAME_POINTER__
     5,040⏰      2,468⏳   1,280k   1,888iop build/bootstrap/fixupobj.com o/asan/third_party/ggml/ggml.o
35,445,258⏰ 28,825,649⏳ 410,616k  64,112iop o/third_party/gcc/bin/x86_64-linux-musl-g++ -msse3 -fno-math-errno -fno-ident -fno-common -fno-gnu-unique -fstrict-aliasing -fstrict-overflow -fno-semantic-interposition -mno-red-zone -mno-tls-direct-seg-refs -fsanitize=address -Wall -Werror -fdebug-prefix-map=/home/jm/src/cosmopolitan= -frecord-gcc-switches -fno-omit-frame-pointer -fno-optimize-sibling-calls -mno-omit-leaf-frame-pointer -fno-schedule-insns2 -O2 -DSYSDEBUG -g -gdwarf-4 -gdescribe-dies -fportcosmo -include build/portcosmo.h -DCOSMO -DMODE="asan" -DIMAGE_BASE_VIRTUAL=0x400000 -nostdinc -iquote . -include libc/integral/normalize.inc -Wa,-W -Wa,-I. -Wa,--noexecstack -Wa,--nocompress-debug-sections -msse3 -fno-math-errno -fno-ident -fno-common -fno-gnu-unique -fstrict-aliasing -fstrict-overflow -fno-semantic-interposition -mno-red-zone -mno-tls-direct-seg-refs -fsanitize=address -fno-rtti -fno-exceptions -fuse-cxa-atexit -fno-threadsafe-statics -Wno-int-in-bool-context -Wno-narrowing -Wno-literal-suffix -c -o /home/jm/src/cosmopolitan/o/tmp/o_asan_third_party_radpajama_gptneox_o.7i9ykl/o_asan_third_party_radpajama_gptneox.o third_party/radpajama/gptneox.cc

Linux 6.2.15-300.fc38.x86_64 #1 SMP PREEMPT_DYNAMIC x86_64 GNU/Linux

[jart]

The MODE=dbg build is in a disrupted state right now due to all the recent improvements. It's something that's actively being worked on right now.

[ghost]

Ok. Looks like a working build slipped in there and threw off the bisect. Taking a larger swath of commits I see the breakage goes back much further.

[jart]

I've just pushed a change that should improve the MODE=asan story for you. I've confirmed that this works:

m=asan
make -j16 m=$m o/$m/third_party/ggml/llama.com
o/$m/third_party/ggml/llama.com.dbg -m ggml-model-q4_0-ggjt-v3.bin -p $'Addition can be performed as such:\n\n```c\nint Add(int x, int y) {\n' -r $'```' -v

[jart]

I've hunted down the remaining regressions in ASAN support and pushed some fixes. It's now looking fabulous again.

[jart]

For the record, I think it was the GCC11 upgrade that did this.

[ghost]

Builds here too. Thanks.