riboseinc / retrace

retrace is a versatile security vulnerability / bug discovery tool through monitoring and modifying the behavior of compiled binaries on Linux, OpenBSD/FreeBSD/NetBSD (shared object) and macOS (dynamic library).
Other
60 stars 19 forks source link

Cant build with --enable-v2=yes #363

Closed ghost closed 5 years ago

ghost commented 6 years ago

@jjr840430 @ronaldtse Guys, your integration of nereon to v2 renders it unbuildable! First of all, there is nothing in build section of docs regarding installing the nereon lib. I took me too much time to figure that a. libnereon is needed. b libnereon does not builds since its Cmake-generated make fails on libucl.

Please clean up that mess or at least remove nereon dependency on v2 meanwhile

BR

jjr840430 commented 6 years ago

@ikolomi Can you try to give the following command to build libnereon?

git clone git@github.com:riboseinc/libnereon.git
cd libnereon
mkdir build
cd build
cmake ..
make
make install

I will add the document how to build libnereon in the next pull. Thanks!

ghost commented 6 years ago

I already did - libnereon pulls libucl, right? So it’s ./configure fails...

On Wed, 22 Aug 2018 at 20:37 Jin notifications@github.com wrote:

@ikolomi https://github.com/ikolomi Can you try to give the following command to build libnereon?

git clone git@github.com:riboseinc/libnereon.git cd libnereon mkdir build cd build cmake .. make make install

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/riboseinc/retrace/issues/363#issuecomment-415115790, or mute the thread https://github.com/notifications/unsubscribe-auth/AfMHWwqX-u6-78HAW7Sz7uY4_4dSRT5vks5uTZbNgaJpZM4WIDrD .

jjr840430 commented 6 years ago

What system are you working on? Can you print here error messages while running configure?

ronaldtse commented 6 years ago

In this case, @jjr840430 can you help move out libnereon to a separate branch? Since @ikolomi is still working on it. You can continuously merge the v2 changes in to your branch. Thanks.

@ikolomi are there parts of v2 that can be merged to master? It’s hard to maintain a branch that depends on another branch. Thanks!

jjr840430 commented 6 years ago

@ronaldtse The master branch of retrace was passed by Travis CI with libnereon.

ghost commented 6 years ago

@jjr840430 Its Ubuntu 17.10. The error log: ikolomi@ikolomi-VirtualBox:home/Documents/libnereon$ make Scanning dependencies of target libucl [ 2%] Creating directories for 'libucl' [ 5%] Performing download step (download, verify and extract) for 'libucl' -- Downloading... dst='/home/ikolomi/Documents/libnereon/libucl-prefix/src/0.8.1.tar.gz' timeout='none' -- Using src='https://github.com/vstakhov/libucl/archive/0.8.1.tar.gz' -- Downloading... done -- extracting... src='/home/ikolomi/Documents/libnereon/libucl-prefix/src/0.8.1.tar.gz' dst='/home/ikolomi/Documents/libnereon/libucl' -- extracting... [tar xfz] -- extracting... [analysis] -- extracting... [rename] -- extracting... [clean up] -- extracting... done [ 8%] Performing autogen step for 'libucl' libtoolize: putting auxiliary files in '.'. libtoolize: copying file './ltmain.sh' libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. libtoolize: copying file 'm4/libtool.m4' libtoolize: copying file 'm4/ltoptions.m4' libtoolize: copying file 'm4/ltsugar.m4' libtoolize: copying file 'm4/ltversion.m4' libtoolize: copying file 'm4/lt~obsolete.m4' configure.ac:71: error: possibly undefined macro: AC_MSG_ERROR If this token and others are legitimate, please use m4_pattern_allow. See the Autoconf documentation. configure.ac:77: error: possibly undefined macro: AC_DEFINE autoreconf: /usr/bin/autoconf failed with exit status: 1 CMakeFiles/libucl.dir/build.make:120: recipe for target 'libucl-prefix/src/libucl-stamp/libucl-autogen' failed make[2]: [libucl-prefix/src/libucl-stamp/libucl-autogen] Error 1 CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/libucl.dir/all' failed make[1]: [CMakeFiles/libucl.dir/all] Error 2 Makefile:140: recipe for target 'all' failed make: *** [all] Error 2

@ronaldtse I am not maintaining feature branches - i prepared feature/support_varargs branch and wanted to rebase on master before creating the pull req. after the rebase I`ve got all the changes with autoconf and libnereon for v2 which now does not compile

jjr840430 commented 6 years ago

@ikolomi Did you install automake and libtool packages on your system?

ronaldtse commented 6 years ago

@jjr840430 in this case can we add Ubuntu to the Travis matrix to ensure it always passes? Thanks!

ghost commented 6 years ago

@jjr840430 Of course, these are prereqs for retrace v1

jjr840430 commented 6 years ago

can you try to install pkg-config package?

ghost commented 6 years ago

@jjr840430 Yes, it solved the issue. Kindly update the build section do and make libnereon dependent on pkg-config

ghost commented 6 years ago

@jjr840430 , there is a problem with the produced libretrace_v2.so: LD_PRELOAD=/usr/local/lib/libretrace_v2.so src/v2/examples/printf/printf_exmpl src/v2/examples/printf/printf_exmpl: symbol lookup error: /usr/local/lib/libretrace_v2.so: undefined symbol: dlopen

It seems that the Makefile_linux was not ported correctly to automake. I tried to fix it but was not successful. the -ldl seems to be present on the libretrace_v2_la_LDFLAGS for linux... Could you please take a look?

jjr840430 commented 6 years ago

@ikolomi -ldl is in libretrace_v2_la_LDFLAGS. Also, I can run src/v2/examples/getenv/getenv_exmpl without any issue. Can you try with getenv_exmpl? Thanks!

ghost commented 5 years ago

@jjr840430 I cant fix the linkage issue with dlopen - with your automake commit, on Ubuntu, libretrace_v2.so is not linked with libdl.so , so the dlopen is undefined. Please fix this, as I spent too much time trying to fix it myself. Otherwise, we will have to revert your automake for v2

ghost commented 5 years ago

Original working version (created with Makefile_linux) imports the following:

    linux-vdso.so.1 (0x00007ffe3ddc4000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ff6f60d7000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff6f5ce6000)
    /lib64/ld-linux-x86-64.so.2 (0x00007ff6f6583000)

While automake version imports:

    linux-vdso.so.1 (0x00007ffc2b3f4000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f81610d3000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8160ce2000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f8161596000)
ghost commented 5 years ago

@ronaldtse Hi Ronald Could you kindly help solving the autotools integration issues?

ronaldtse commented 5 years ago

@ikolomi I get the message, @jjr840430 is currently on holiday, he should be able to help once he's back. Thanks!

ghost commented 5 years ago

@ronaldtse Thanks a lot, I’ll continue to push to fix it, but there seems to be something fundamentally wrong with how libtool builds it

jjr840430 commented 5 years ago

@ikolomi Can you push your current working branch? When I'm trying to run the following command, it doesn't have any issue related to dlopen function link.

$ LD_PRELOAD=/usr/local/lib/libretrace_v2.so src/v2/examples/getenv/getenv_exmpl USER
$ 'USER' = 'dev'

I'm not sure why stdout doesn't show any tracing info but will try to solve dlopen problem at first. Thanks.

ghost commented 5 years ago

@jjr840430 , the issue happens on the master branch. There was a small issue with undefined symbol retrace_as_init_late in linux compilation , but i just pushed a fix for it. My research suggests that this is a specific Debian (and its derivatives like Ubuntu). I suggest that you try to build on ubuntu and try getenv_exmpl. You should have either a sigfault or "dlopen is undefined" error

pablo-mendoza commented 5 years ago

retrace v2 is crashing in ubuntu with any program when build without --enable-debug, if I build with --enable-debug then it works (which makes this hard to debug ...)

So, I figure out that building that -O2 or -O1 makes retrace crash on everything, abd gdb doesn't show any good output. It only works with -O0 (or no -O flag)

pablo-mendoza commented 5 years ago

I put in a fix that basically always build with -O0 since I couldn't exactly figure out why this was crashing.

https://github.com/riboseinc/retrace/pull/386

pablo-mendoza commented 5 years ago

This seems to be working now, please reopen if it still failing.