search

avast / retdec

RetDec is a retargetable machine-code decompiler based on LLVM.
https://retdec.com/
MIT License
8k stars 946 forks source link

Problem with Compiling on macOS 10.11 #560

Closed humanitiesclinic closed 5 years ago

humanitiesclinic commented 5 years ago 
$ make -j2
[  1%] Built target capstone-project
[  1%] Built target jsoncpp-project
[  2%] Built target openssl
[  3%] Built target llvm-project
[  4%] Built target rapidjson-project
[  4%] Built target tinyxml2-project
[  5%] Built target elfio-project
[  6%] Built target libdwarf-project
[  7%] Built target pelib-project
[  8%] Performing build step for 'yaracpp-project'
Scanning dependencies of target yaramod-project
[  8%] Creating directories for 'yaramod-project'
CMake Error at /Users/user/AppInstalls/zDEVELOP/retdec/build/external/src/yaracpp-project-stamp/yaracpp-project-build-Release.cmake:16 (message):
  Command failed: 2

   '/Applications/Xcode.app/Contents/Developer/usr/bin/make'

  See also

    /Users/user/AppInstalls/zDEVELOP/retdec/build/external/src/yaracpp-project-stamp/yaracpp-project-build-*.log

make[2]: *** [external/src/yaracpp-project-stamp/yaracpp-project-build] Error 1
make[1]: *** [deps/yaracpp/CMakeFiles/yaracpp-project.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[  8%] Performing download step (download, verify and extract) for 'yaramod-project'
-- yaramod-project download command succeeded.  See also /Users/user/AppInstalls/zDEVELOP/retdec/build/external/src/yaramod-project-stamp/yaramod-project-download-*.log
[  8%] No update step for 'yaramod-project'
[  8%] No patch step for 'yaramod-project'
[  8%] Performing configure step for 'yaramod-project'
CMake Error at /Users/user/AppInstalls/zDEVELOP/retdec/build/external/src/yaramod-project-stamp/yaramod-project-configure-Release.cmake:16 (message):
  Command failed: 1

   '/usr/local/Cellar/cmake/3.13.3/bin/cmake' '-DCMAKE_BUILD_TYPE=Release' '-DCMAKE_C_COMPILER=/Library/Developer/CommandLineTools/usr/bin/cc' '-DCMAKE_CXX_COMPILER=/Library/Developer/CommandLineTools/usr/bin/c++' '-GUnix Makefiles' '/Users/user/AppInstalls/zDEVELOP/retdec/build/external/src/yaramod-project'

  See also

    /Users/user/AppInstalls/zDEVELOP/retdec/build/external/src/yaramod-project-stamp/yaramod-project-configure-*.log

make[2]: *** [external/src/yaramod-project-stamp/yaramod-project-configure] Error 1
make[1]: *** [deps/yaramod/CMakeFiles/yaramod-project.dir/all] Error 2
make: *** [all] Error 2

I checked and saw that this binary exists: /Applications/Xcode.app/Contents/Developer/usr/bin/make

Apologies as I am not familiar with cmake syntax.

Any suggestions?

silverbacknet commented 5 years ago

What's in the log it pointed to, /Users/user/AppInstalls/zDEVELOP/retdec/build/external/src/yaramod-project-stamp/yaramod-project-configure-*.log ?

s3rvac commented 5 years ago

Apart from the file that @silverbacknet has mentioned, please also attach

/Users/user/AppInstalls/zDEVELOP/retdec/build/external/src/yaracpp-project-stamp/yaracpp-project-build-*.log

So, to sum up, we will need the following four log files:

/Users/user/AppInstalls/zDEVELOP/retdec/build/external/src/yaracpp-project-stamp/yaracpp-project-build-err.log
/Users/user/AppInstalls/zDEVELOP/retdec/build/external/src/yaracpp-project-stamp/yaracpp-project-build-out.log
/Users/user/AppInstalls/zDEVELOP/retdec/build/external/src/yaramod-project-stamp/yaramod-project-configure-err.log
/Users/user/AppInstalls/zDEVELOP/retdec/build/external/src/yaramod-project-stamp/yaramod-project-configure-out.log

Please, attach them in a single archive.

humanitiesclinic commented 5 years ago

yaracpp-project-build-err.log yaracpp-project-build-out.log

yaracpp-project-configure-out.log

Screen Shot 2019-04-29 at 2 45 12 PM

One of the files has zero bytes.

Oh oops, u asked for a single archive. Just saw that. Here it is if u prefer:

Archive.zip

silverbacknet commented 5 years ago

Another autoreconf error. Check this report: https://github.com/avast/retdec/issues/291

humanitiesclinic commented 5 years ago

I tried all the suggestions in #291: (1) installed autotools (2) checked if aclocal is installed - yes (3) make sure autotools, automaker, aclocal, libtool are all available in $PATH (4) installed gnu-getopt

Still having issues:

Users-Air:build user$ make -j2
[  1%] Built target jsoncpp-project
[  1%] Built target capstone-project
[  3%] Built target llvm-project
[  3%] Built target openssl
[  4%] Built target tinyxml2-project
[  4%] Built target rapidjson-project
[  5%] Built target libdwarf-project
[  6%] Built target elfio-project
[  7%] Performing build step for 'yaracpp-project'
[  8%] Built target pelib-project
[  9%] Built target yaramod-project
Scanning dependencies of target whereami
[ 10%] Building C object deps/whereami/CMakeFiles/whereami.dir/whereami.c.o
[ 10%] Linking C static library libwhereami.a
[ 10%] Built target whereami
Scanning dependencies of target retdec-pdbparser
[ 10%] Building CXX object src/pdbparser/CMakeFiles/retdec-pdbparser.dir/pdb_file.cpp.o
[ 10%] Building CXX object src/pdbparser/CMakeFiles/retdec-pdbparser.dir/pdb_symbols.cpp.o
[ 11%] Building CXX object src/pdbparser/CMakeFiles/retdec-pdbparser.dir/pdb_types.cpp.o
[ 11%] Building CXX object src/pdbparser/CMakeFiles/retdec-pdbparser.dir/pdb_utils.cpp.o
[ 11%] Linking CXX static library libretdec-pdbparser.a
[ 11%] Built target retdec-pdbparser
Scanning dependencies of target retdec-utils
[ 11%] Building CXX object src/utils/CMakeFiles/retdec-utils.dir/address.cpp.o
[ 11%] Building CXX object src/utils/CMakeFiles/retdec-utils.dir/alignment.cpp.o
[ 11%] Building CXX object src/utils/CMakeFiles/retdec-utils.dir/byte_value_storage.cpp.o
CMake Error at /Users/user/AppInstalls/zDEVELOP/retdec/build/external/src/yaracpp-project-stamp/yaracpp-project-build-Release.cmake:16 (message):
  Command failed: 2

   '/Applications/Xcode.app/Contents/Developer/usr/bin/make'

  See also

    /Users/user/AppInstalls/zDEVELOP/retdec/build/external/src/yaracpp-project-stamp/yaracpp-project-build-*.log

make[2]: *** [external/src/yaracpp-project-stamp/yaracpp-project-build] Error 1
make[1]: *** [deps/yaracpp/CMakeFiles/yaracpp-project.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 11%] Building CXX object src/utils/CMakeFiles/retdec-utils.dir/binary_path.cpp.o
[ 12%] Building CXX object src/utils/CMakeFiles/retdec-utils.dir/conversion.cpp.o
[ 12%] Building CXX object src/utils/CMakeFiles/retdec-utils.dir/dynamic_buffer.cpp.o
[ 12%] Building CXX object src/utils/CMakeFiles/retdec-utils.dir/file_io.cpp.o
[ 12%] Building CXX object src/utils/CMakeFiles/retdec-utils.dir/filesystem_path.cpp.o
[ 12%] Building CXX object src/utils/CMakeFiles/retdec-utils.dir/math.cpp.o
[ 12%] Building CXX object src/utils/CMakeFiles/retdec-utils.dir/memory.cpp.o
[ 12%] Building CXX object src/utils/CMakeFiles/retdec-utils.dir/string.cpp.o
[ 12%] Building CXX object src/utils/CMakeFiles/retdec-utils.dir/system.cpp.o
[ 12%] Building CXX object src/utils/CMakeFiles/retdec-utils.dir/time.cpp.o
[ 13%] Linking CXX static library libretdec-utils.a
[ 13%] Built target retdec-utils
make: *** [all] Error 2

yaracpp-project-build-err.log yaracpp-project-build-out.log

yaracpp-project-configure-out.log

humanitiesclinic commented 5 years ago

Any suggestions, anyone?

metthal commented 5 years ago

According to output in yaracpp-project-build-err.log, try installing libtool or make sure it's in your PATH.

humanitiesclinic commented 5 years ago

Still the same:

$ make -j2
[  1%] Built target jsoncpp-project
[  1%] Built target capstone-project
[  2%] Built target llvm-project
[  3%] Built target openssl
[  4%] Built target rapidjson-project
[  4%] Built target tinyxml2-project
[  5%] Built target elfio-project
[  6%] Built target libdwarf-project
[  7%] Built target pelib-project
[  8%] Performing build step for 'yaracpp-project'
[  9%] Built target yaramod-project
[ 10%] Built target whereami
[ 11%] Built target retdec-pdbparser
[ 13%] Built target retdec-utils
[ 13%] Built target retdec-ar-extractor
Scanning dependencies of target retdec-capstone2llvmir
[ 13%] Building CXX object src/capstone2llvmir/CMakeFiles/retdec-capstone2llvmir.dir/arm/arm_init.cpp.o
[ 13%] Building CXX object src/capstone2llvmir/CMakeFiles/retdec-capstone2llvmir.dir/arm/arm.cpp.o
CMake Error at /Users/user/AppInstalls/zDEVELOP/retdec/build/external/src/yaracpp-project-stamp/yaracpp-project-build-Release.cmake:16 (message):
  Command failed: 2

   '/Applications/Xcode.app/Contents/Developer/usr/bin/make'

  See also

    /Users/user/AppInstalls/zDEVELOP/retdec/build/external/src/yaracpp-project-stamp/yaracpp-project-build-*.log

make[2]: *** [external/src/yaracpp-project-stamp/yaracpp-project-build] Error 1
make[1]: *** [deps/yaracpp/CMakeFiles/yaracpp-project.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 13%] Building CXX object src/capstone2llvmir/CMakeFiles/retdec-capstone2llvmir.dir/mips/mips_init.cpp.o
[ 14%] Building CXX object src/capstone2llvmir/CMakeFiles/retdec-capstone2llvmir.dir/mips/mips.cpp.o
[ 14%] Building CXX object src/capstone2llvmir/CMakeFiles/retdec-capstone2llvmir.dir/powerpc/powerpc_init.cpp.o
[ 14%] Building CXX object src/capstone2llvmir/CMakeFiles/retdec-capstone2llvmir.dir/powerpc/powerpc.cpp.o
[ 14%] Building CXX object src/capstone2llvmir/CMakeFiles/retdec-capstone2llvmir.dir/x86/x86_init.cpp.o
[ 14%] Building CXX object src/capstone2llvmir/CMakeFiles/retdec-capstone2llvmir.dir/x86/x86.cpp.o
[ 14%] Building CXX object src/capstone2llvmir/CMakeFiles/retdec-capstone2llvmir.dir/capstone2llvmir_impl.cpp.o
[ 14%] Building CXX object src/capstone2llvmir/CMakeFiles/retdec-capstone2llvmir.dir/capstone2llvmir.cpp.o
[ 14%] Building CXX object src/capstone2llvmir/CMakeFiles/retdec-capstone2llvmir.dir/exceptions.cpp.o
[ 14%] Building CXX object src/capstone2llvmir/CMakeFiles/retdec-capstone2llvmir.dir/llvmir_utils.cpp.o
[ 15%] Linking CXX static library libretdec-capstone2llvmir.a
[ 15%] Built target retdec-capstone2llvmir
make: *** [all] Error 2

This is what I get when I invoke libtool:

$ libtool
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: no output file specified (specify with -o output)
Usage: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool -static [-] file [...] [-filelist listfile[,dirname]] [-arch_only arch] [-sacLT] [-no_warning_for_no_symbols]
Usage: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool -dynamic [-] file [...] [-filelist listfile[,dirname]] [-arch_only arch] [-o output] [-install_name name] [-compatibility_version #] [-current_version #] [-seg1addr 0x#] [-segs_read_only_addr 0x#] [-segs_read_write_addr 0x#] [-seg_addr_table <filename>] [-seg_addr_table_filename <file_system_path>] [-all_load] [-noall_load]

It exists. But it's throwing an error. Could that be the problem?

metthal commented 5 years ago

That's indeed weird. Can you please try this in some temporary folder?

git clone https://github.com/avast/yaracpp.git
cd yaracpp
git checkout macos-build
mkdir build && cd build
cmake ..
cmake --build .
humanitiesclinic commented 5 years ago

Tried that.

Output:

Users-Air:zDEVELOP user$ git clone https://github.com/avast/yaracpp.git
Cloning into 'yaracpp'...
remote: Enumerating objects: 42, done.
remote: Counting objects: 100% (42/42), done.
remote: Compressing objects: 100% (24/24), done.
remote: Total 239 (delta 16), reused 29 (delta 9), pack-reused 197
Receiving objects: 100% (239/239), 49.83 KiB | 197.00 KiB/s, done.
Resolving deltas: 100% (103/103), done.
Users-Air:zDEVELOP user$ cd yaracpp
Users-Air:yaracpp user$ git checkout macos-build
Branch 'macos-build' set up to track remote branch 'macos-build' from 'origin'.
Switched to a new branch 'macos-build'
Users-Air:yaracpp user$ mkdir build && cd build
Users-Air:build user$ cmake ..
-- The CXX compiler identification is AppleClang 7.3.0.7030031
-- The C compiler identification is AppleClang 7.3.0.7030031
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- ARGN is: Threads_FOUND
-- FPHSA_REQUIRED_VARS is: Threads_FOUND
-- Threads_FOUND, came to else
-- Found Threads: TRUE  
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/user/AppInstalls/zDEVELOP/yaracpp/build
Users-Air:build user$ cmake --build .
Scanning dependencies of target yara
[  7%] Creating directories for 'yara'
[ 14%] Performing download step (download, verify and extract) for 'yara'
-- Downloading...
   dst='/Users/user/AppInstalls/zDEVELOP/yaracpp/build/deps/yara/src/yara.zip'
   timeout='none'
-- Using src='https://github.com/avast-tl/yara/archive/102811d0ec8751307a5e6996929e11fcfcee3b30.zip'
-- verifying file...
       file='/Users/user/AppInstalls/zDEVELOP/yaracpp/build/deps/yara/src/yara.zip'
-- Downloading... done
-- extracting...
     src='/Users/user/AppInstalls/zDEVELOP/yaracpp/build/deps/yara/src/yara.zip'
     dst='/Users/user/AppInstalls/zDEVELOP/yaracpp/build/deps/yara/src/yara'
-- extracting... [tar xfz]
-- extracting... [analysis]
-- extracting... [rename]
-- extracting... [clean up]
-- extracting... done
[ 21%] No patch step for 'yara'
[ 28%] No update step for 'yara'
[ 35%] No configure step for 'yara'
[ 42%] Performing configure-linux step for 'yara'
configure.ac:20: installing 'build-aux/ar-lib'
configure.ac:20: installing 'build-aux/compile'
configure.ac:29: installing 'build-aux/config.guess'
configure.ac:29: installing 'build-aux/config.sub'
configure.ac:8: installing 'build-aux/install-sh'
configure.ac:8: installing 'build-aux/missing'
Makefile.am: installing 'build-aux/depcomp'
parallel-tests: installing 'build-aux/test-driver'
libyara/Makefile.am:115: error: Libtool library used but 'LIBTOOL' is undefined
libyara/Makefile.am:115:   The usual way to define 'LIBTOOL' is to add 'LT_INIT'
libyara/Makefile.am:115:   to 'configure.ac' and run 'aclocal' and 'autoconf' again.
libyara/Makefile.am:115:   If 'LT_INIT' is in 'configure.ac', make sure
libyara/Makefile.am:115:   its definition is in aclocal's search path.
configure.ac: installing 'build-aux/ylwrap'
autoreconf: automake failed with exit status: 1
make[2]: *** [deps/yara/src/yara-stamp/yara-configure-linux] Error 1
make[1]: *** [deps/CMakeFiles/yara.dir/all] Error 2
make: *** [all] Error 2

In the same shell/terminal window: Users-Air:build user$ libtool

error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool: no output file specified (specify with -o output)
Usage: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool -static [-] file [...] [-filelist listfile[,dirname]] [-arch_only arch] [-sacLT] [-no_warning_for_no_symbols]
Usage: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/libtool -dynamic [-] file [...] [-filelist listfile[,dirname]] [-arch_only arch] [-o output] [-install_name name] [-compatibility_version #] [-current_version #] [-seg1addr 0x#] [-segs_read_only_addr 0x#] [-segs_read_write_addr 0x#] [-seg_addr_table <filename>] [-seg_addr_table_filename <file_system_path>] [-all_load] [-noall_load]
metthal commented 5 years ago

So you are not even able to build yaracpp, let's go even deeper. Try building YARA itself without any additional layer above it.

git clone https://github.com/avast/yara.git
cd yara
./bootstrap.sh
./configure
make
humanitiesclinic commented 5 years ago

I got a problem at ./bootstrap.sh

Users-Air:zDEVELOP user$ git clone https://github.com/avast/yara.git
Cloning into 'yara'...
remote: Enumerating objects: 14448, done.
remote: Total 14448 (delta 0), reused 0 (delta 0), pack-reused 14448
Receiving objects: 100% (14448/14448), 17.89 MiB | 1.32 MiB/s, done.
Resolving deltas: 100% (10386/10386), done.
Users-Air:zDEVELOP user$ cd yara
Users-Air:yara user$ ./bootstrap.sh
configure.ac:20: installing 'build-aux/ar-lib'
configure.ac:20: installing 'build-aux/compile'
configure.ac:29: installing 'build-aux/config.guess'
configure.ac:29: installing 'build-aux/config.sub'
configure.ac:8: installing 'build-aux/install-sh'
configure.ac:8: installing 'build-aux/missing'
Makefile.am: installing 'build-aux/depcomp'
parallel-tests: installing 'build-aux/test-driver'
libyara/Makefile.am:115: error: Libtool library used but 'LIBTOOL' is undefined
libyara/Makefile.am:115:   The usual way to define 'LIBTOOL' is to add 'LT_INIT'
libyara/Makefile.am:115:   to 'configure.ac' and run 'aclocal' and 'autoconf' again.
libyara/Makefile.am:115:   If 'LT_INIT' is in 'configure.ac', make sure
libyara/Makefile.am:115:   its definition is in aclocal's search path.
configure.ac: installing 'build-aux/ylwrap'
autoreconf: automake failed with exit status: 1
metthal commented 5 years ago

From what I googled, could you please try to install libtool from brew and make sure it precedes your current libtool in PATH?

humanitiesclinic commented 5 years ago

u mean "brew install libtool"?

humanitiesclinic commented 5 years ago

i just installed lib tool using

brew install libtool How do I know where the just-installed libtool is (vs the one that already exists)?

metthal commented 5 years ago

I don't have macOS so I am not sure where brew installs its binaries. Try to find it in your filesystem.

humanitiesclinic commented 5 years ago

ok i managed to compile it after installing lib tool directly from brew. Thank you very much.

Btw, is there only one binary produced, which is retdec-bin2llvmir?

s3rvac commented 5 years ago

There should be many binaries and scripts after the installation, including e.g. $INSTALL_DIR/bin/retdec-decompiler.py and $INSTALL_DIR/bin/retdec-fileinfo.