Make fails with GCC 7.3 on Ubuntu 18.04

[mrigger]

Hi Stephen! After compiling liballocs according to https://github.com/stephenrkell/liballocs/issues/10, I stumbled into the following issue when compiling libcrunch:

cc -DLIBCRUNCH_LONG_SIZE -DLIBCRUNCH_NO_DENORM_BOUNDS  -I//liballocs/include -I//liballocs/src -I//liballocs/contrib/trap-syscalls/libsystrap -I//liballocs/contrib/librunt/include -I//liballocs/contrib/libdlbind/include -std=gnu99 -g -fPIC -I../include -O3 -flto -ffat-lto-objects -DNDEBUG -fno-omit-frame-pointer   -c -o libcrunch.o libcrunch.c
libcrunch.c: In function 'handle_sigsegv':
libcrunch.c:701:48: error: dereferencing pointer to incomplete type 'struct ucontext'
  void *faulting_code_address = (void*) ucontext->uc_mcontext.gregs[REG_RIP];
                                                ^~
In file included from ../include/libcrunch.h:48:0,
                 from libcrunch.c:26:
libcrunch.c: At top level:
../include/libcrunch_cil_inlines.h:1424:4: warning: 'warnx_pure' is static but used in inline function '__fetch_bounds_from_shadow_space' which is not static
    warnx_pure("Fetched invalid bounds for %p (loaded from %p)", ptr, loaded_from);
    ^~~~~~~~~~
../include/libcrunch_cil_inlines.h:1150:3: warning: 'warnx_pure' is static but used in inline function '__secondary_check_derive_ptr' which is not static
   warnx_pure("Created one-past pointer at %p: %p (base %p, size %lu)",
   ^~~~~~~~~~
../include/libcrunch_cil_inlines.h:1127:4: warning: 'warnx_pure' is static but used in inline function '__secondary_check_derive_ptr' which is not static
    warnx_pure("Went back in bounds at %p: %p (base %p, size %lu)",
    ^~~~~~~~~~
<builtin>: recipe for target 'libcrunch.o' failed
make[1]: Leaving directory '/libcrunch/src'
make[1]: *** [libcrunch.o] Error 1
Makefile:5: recipe for target 'src' failed
make: *** [src] Error 2

I extended the liballocs Dockerfile that is referenced above by the following line:

RUN export LIBALLOCS=`pwd`/liballocs && git clone https://github.com/stephenrkell/libcrunch.git && cd libcrunch && make

Do you have any ideas on how to fix this?

[stephenrkell]

Hi Manuel. Yes, an educated guess: replace mention of "struct ucontext" with "ucontext_t". I should have done this already... hmm, I probably have the changes unpushed somewhere and will take a look, but if you try it do let me know how it goes. -- Sent from my Android device with K-9 Mail. Please excuse my brevity.

[stephenrkell]

Indeed I had that fix on a private branch. I've just pushed a merge, so try now...

[mrigger]

I can confirm that your changes fixed the compilation issues! Thank you!

It seems that libcrunch, after being compiled, crashes during run time:

frontend c, config default, cases: like-a hello-stack fail-voidptrptr-invalid hello hello-stackactual hello-errno fail-sloppy-gpcot fail-voidptrptr-strict hello-sizeofness function-refines hello-qualified-char hello-static hello-stubgen hello-array hello-void hello-funptr pointer-degree sloppy-gpcot fail-va_arg hello-heap va_arg hello-incomplete hello-indirect hello-union lazy-typing fail-funptr voidptrptr
LD_PRELOAD="/libcrunch/src/libcrunch_preload.so" ./hello
liballocs: process name hello, pid 719
in executable mapping 0x56516bebe000-0x56516bebf000, we think the last section ends at 0x56516bebe94d
in executable mapping 0x7f9784528000-0x7f978452b000, we think the last section ends at 0x7f9784529be9
in executable mapping 0x7f978472c000-0x7f9784913000, we think the last section ends at 0x7f97848c8882
Segmentation fault (core dumped)
make[5]: *** [_onlyrun-hello] Error 139
make[4]: *** [run-hello] Error 2
../../Makefile:314: recipe for target '_onlyrun-hello' failed
make[5]: Leaving directory '/libcrunch/test/hello/c+default'
Makefile:329: recipe for target 'run-hello' failed
make[4]: Leaving directory '/libcrunch/test'
make[3]: *** [cleanrun-hello] Error 2
make[2]: *** [sanity-tests] Error 2
Makefile:337: recipe for target 'cleanrun-hello' failed
make[3]: Leaving directory '/libcrunch/test'
Makefile:71: recipe for target 'sanity-tests' failed
rm vsimplemem/vsimplemem.cmo alloclocals/alloclocals.cmo trumptr/trumptr.cmx crunchbound/crunchbound.cmo ptrintarith/ptrintarith.cmo vsimplemem/vsimplemem.cmx dumpreftypes/dumpreftypes.cmo crunchbound/crunchbound.cmx shadowprov/shadowprov.cmo shadow/shadow.cmo vsimpleaddr/vsimpleaddr.cmo dumpreftypes/dumpreftypes.cmx alloclocals/alloclocals.cmx vsimpleaddr/vsimpleaddr.cmx trumptr/trumptr.cmo ptrintarith/ptrintarith.cmx
make[2]: Leaving directory '/libcrunch/frontend/c'
Makefile:5: recipe for target 'c' failed
make[1]: *** [c] Error 2
make[1]: Leaving directory '/libcrunch/frontend'
Makefile:9: recipe for target 'frontend' failed
make: *** [frontend] Error 2
The command '/bin/sh -c export LIBALLOCS=`pwd`/liballocs && git clone https://github.com/stephenrkell/libcrunch.git && cd libcrunch && make' returned a non-zero code: 2
Makefile:16: recipe for target 'build' failed
make: *** [build] Error 2

[stephenrkell]

Hmm. At first glance this could be lots of things. So I'll give you the generic debugging instructions first. If from the "test" directory you do

make FRONTEND=c gdbrun-hello

... you should be able to get a backtrace at the site of the crash. This doesn't work if the crash is very early during startup. Then you'll have to do

LIBALLOCS_DELAY_STARTUP=1 make FRONTEND=c run-hello

... and using the PID it prints out, attach a debugger within ten seconds (gdb -p $PID).

On further thought I think in this case, the problem is an instruction in your libc that the x86 decoder in liballocs (actually in libsystrap) cannot decode. This happens when someone tries with a newer C library. The problem happens at a time when attempting to report an error instead causes a segfault (execute permission has been temporarily turned off on libc's text segment).

I've just merged another pull request in stephenrkell/trap-syscalls. So I suggest you do something like

cd $LIBALLOCS/contrib/trap-syscalls
git pull origin master
git submodule update --init
rm -f contrib/config.mk
make -C contrib
cd $LIBALLOCS/contrib
make build-libsystrap
cd $LIBALLOCS
make
cd $LIBCRUNCH
make

That should advance you to the next problem -- probably not out of the woods yet. My guess is you're using the same (or similar) distribution as our current research intern Guillaume, who has fixed some problems like this (hence why I predict you'll have one or two more). He is in the process of pull-requesting his changes, you could just wait for those to appear (within a day or two I hope). I'll comment back when it's done.

[mrigger]

Thanks for the detailed feedback! I think I'll wait until Guillaume has pushed the changes (no hurries), after which I'll try again.

[mrigger]

Has any progress been made on this? :blush:

[stephenrkell]

Sorry for the delay. I've just merged the changes. Also I've just re-run an Ubuntu 18.04 build test of liballocs and it finishes OK (for what that's worth!). Let me know how you get on!

[stephenrkell]

This issue is now stale, so closing... am about to add a Dockerfile-based build test much like liballocs's (thanks Manuel for starting the tradition) and this will cover Ubuntu 18.x. Of course, now 20.x is a problem. :-)