BinaryAnalysisPlatform / bap

Binary Analysis Platform
MIT License
2.05k stars 271 forks source link

Installation instructions Macos with Brew #925

Closed JoranHonig closed 5 years ago

JoranHonig commented 5 years ago

Hi,

I wanted to start playing around with bap for a bit, but unfortunately a fresh install of bap and ocaml on my machine didn't go so smoothly.

There seem to be two problems during the installation (opam depext --install bap):

#=== ERROR while compiling conf-bap-llvm.1.3 ==================================#
# context     2.0.3 | macos/x86_64 | ocaml-base-compiler.4.05.0 | https://opam.ocaml.org#9c708876
# path        ~/.opam/4.05.0/.opam-switch/build/conf-bap-llvm.1.3
# command     ~/.opam/opam-init/hooks/sandbox.sh build ocaml find-llvm.ml
# exit-code   1
# env-file    ~/.opam/log/conf-bap-llvm-11562-264e0c.env
# output-file ~/.opam/log/conf-bap-llvm-11562-264e0c.out
### output ###
# [...]
# command which llvm-config failed: exit status 1
# command ls /usr/local/Cellar | grep 3.8 failed: exit status 1
# command which llvm-config-mp-3.8 failed: exit status 1
# command which llvm-config-3.8 failed: exit status 1
# command which llvm-config-38 failed: exit status 1
# command which llvm-config failed: exit status 1
# command ls /usr/local/Cellar | grep 3.4 failed: exit status 1
# command which llvm-config-mp-3.4 failed: exit status 1
# command which llvm-config-3.4 failed: exit status 1
# command which llvm-config-34 failed: exit status 1
# command which llvm-config failed: exit status 1
# LLVM not found

#=== ERROR while compiling bap-objdump.1.5.0 ==================================#
# context     2.0.3 | macos/x86_64 | ocaml-base-compiler.4.05.0 | https://opam.ocaml.org#9c708876
# path        ~/.opam/4.05.0/.opam-switch/build/bap-objdump.1.5.0
# command     ~/.opam/opam-init/hooks/sandbox.sh build ./configure --prefix=/Users/walker/.opam/4.05.0 --enable-objdump --objdump-targets=[] --objdump-path=
# exit-code   1
# env-file    ~/.opam/log/bap-objdump-11562-cb921b.env
# output-file ~/.opam/log/bap-objdump-11562-cb921b.out
### output ###
# configure options:
# [...]
#   --sharedstatedir dir       Modifiable architecture-independent data [$prefix/com]
#   --sysconfdir dir           Read-only single-machine data [$prefix/etc]
#   --libexecdir dir           Program executables [$exec_prefix/libexec]
#   --sbindir dir              System admin executables [$exec_prefix/sbin]
#   --bindir dir               User executables [$exec_prefix/bin]
#   --exec-prefix dir          Install architecture-dependent files in dir [$prefix]
#   --prefix dir               Install architecture-independent files dir [/usr/local]
#   --print-hidden Print       even non-printable variable. (debug)  [false]
#   -help                      Display this list of options
#   --help                     Display this list of options
# 

Some prior issue (https://github.com/ocaml/opam-repository/issues/7575) indicates that the first problem is caused by brew not properly settingup the path variable. Adding /usr/local/Cellar/llvm@6/6.0.1_1 to my path variable makes compilation of conf-bap-llvm succeed but gives a new error:

#=== ERROR while compiling bap-llvm.1.5.0 =====================================#
# context     2.0.3 | macos/x86_64 | ocaml-base-compiler.4.05.0 | https://opam.ocaml.org#9c708876
# path        ~/.opam/4.05.0/.opam-switch/build/bap-llvm.1.5.0
# command     ~/.opam/opam-init/hooks/sandbox.sh build make
# exit-code   2
# env-file    ~/.opam/log/bap-llvm-74651-6aa130.env
# output-file ~/.opam/log/bap-llvm-74651-6aa130.out
### output ###
# [...]
# In file included from /usr/local/Cellar/llvm@6/6.0.1_1/include/llvm/ADT/StringRef.h:13:
# In file included from /usr/local/Cellar/llvm@6/6.0.1_1/include/llvm/ADT/STLExtras.h:20:
# In file included from /usr/local/Cellar/llvm@6/6.0.1_1/include/llvm/ADT/Optional.h:22:
# In file included from /usr/local/Cellar/llvm@6/6.0.1_1/include/llvm/Support/type_traits.h:19:
# In file included from /usr/local/Cellar/llvm@6/6.0.1_1/include/c++/v1/utility:202:
# In file included from /usr/local/Cellar/llvm@6/6.0.1_1/include/c++/v1/cstring:61:
# /usr/local/Cellar/llvm@6/6.0.1_1/include/c++/v1/string.h:61:15: fatal error: 'string.h' file not found
# #include_next <string.h>
#               ^~~~~~~~~~
# 1 error generated.
# Command exited with code 1.
# make: *** [build] Error 1

It's unclear on which version of llvm bap actually depends:

The packages you requested declare the following system dependencies. Please
make sure they are installed before retrying:
    llvm@6

Is reported when trying to execute opam depext --install conf-bap-llvm While the troubleshooting page indicates support for version 3.4 and 3.8 of llvm. Could this second issue in bap-llvm be caused by a version mismatch?

The second issue (relating to objdump) has also recently been reported in #921, the reporter describes using a development version, which is not entirely clear to me. Is anyone able to elaborate on the action required to solve this problem?

Once I've been able to do an install of bap I'm happy to write a piece of documentation for the readme / wiki, with install instructions for how to install bap w/ brew, if that's appreciated

ivg commented 5 years ago

First off all, thanks for a nice bug report :)

The problem with the LLVM actually indicates that your XCode is either broken, out of date, or just missing (especially the command line tools). We, in house, usually rely in on macports, which require XCode to be properly set up, so we usually don't get that far.

This link might give you some insights, on how to set things up.

ivg commented 5 years ago

Also, if you're on Mojave, then this could be also relevant.

JoranHonig commented 5 years ago

Thanks for the quick response!

Fixing my xcode installation has indeed solved the first problem (installing conf-bap-llvm and bap-llvm)

However the second problem (installing bap-objdump) remains. I've included the output file mentioned in the error output:

none: option '--objdump-path' needs an argument.
configure options:
  --override var+val         Override any configuration variable.
  --enable-tests             Compile tests executable and library and run them [default: disabled]
  --disable-tests            Compile tests executable and library and run them [default: disabled]
  --enable-docs              Create documentations [default: enabled]
  --disable-docs             Create documentations [default: enabled]
  --enable-objdump           Build objdump plugin [default: disabled]
  --disable-objdump          Build objdump plugin [default: disabled]
  --enable-everything        Build every feature by default [default: disabled]
  --disable-everything       Build every feature by default [default: disabled]
  --with-cc-optimization str C compiler optimization level [-O2]
  --with-cxxlibs str         c++ library [-lc++]
  --with-cxx str             c++ compiler [/usr/bin/c++]
  --objdump-path str         A path to [g]objdump binary
  --objdump-targets str      A list (OCaml syntax) of supported targets
  --plugindir str            A default folder for plugins
  --enable-profile           Turn ocaml profile flag on [default: disabled]
  --disable-profile          Turn ocaml profile flag on [default: disabled]
  --enable-debug             Turn ocaml debug flag on [default: enabled]
  --disable-debug            Turn ocaml debug flag on [default: enabled]
  --destdir dir              Prepend a path when installing package
  --psdir dir                PS documentation [$docdir]
  --pdfdir dir               PDF documentation [$docdir]
  --dvidir dir               DVI documentation [$docdir]
  --htmldir dir              HTML documentation [$docdir]
  --docdir dir               Documentation root [$datarootdir/doc/$pkg_name]
  --mandir dir               Man documentation [$datarootdir/man]
  --localedir dir            Locale-dependent data [$datarootdir/locale]
  --infodir dir              Info documentation [$datarootdir/info]
  --datadir dir              Read-only architecture-independent data [$datarootdir]
  --datarootdir dir          Read-only arch-independent data root [$prefix/share]
  --libdir dir               Object code libraries [$exec_prefix/lib]
  --localstatedir dir        Modifiable single-machine data [$prefix/var]
  --sharedstatedir dir       Modifiable architecture-independent data [$prefix/com]
  --sysconfdir dir           Read-only single-machine data [$prefix/etc]
  --libexecdir dir           Program executables [$exec_prefix/libexec]
  --sbindir dir              System admin executables [$exec_prefix/sbin]
  --bindir dir               User executables [$exec_prefix/bin]
  --exec-prefix dir          Install architecture-dependent files in dir [$prefix]
  --prefix dir               Install architecture-independent files dir [/usr/local]
  --print-hidden Print       even non-printable variable. (debug)  [false]
  -help                      Display this list of options
  --help                     Display this list of options

I'm sure that objdump is installed which objdump returns /usr/bin/objdump

ivg commented 5 years ago

Nice. So first of all, the bap-objdump package is not very important, especially on a Mac OS X, so you can start playing with BAP right now.

Concerning the problem, our conf-binutils script is expecting the gobdump tool on Darwin, not objdump which is an overhauled llvm-objdump, which is not an objdump at all. Well, at least in my installation, in your system it could be overridden by a true objdump. If it is true that homebrew provides objdump under the objdump name, then we can update our conf-binutils script to reflect that fact.

And in any case, it looks like that the conf-binutils package is not failing gracefully, as it should actually stop with an error, instead of delaying the failure to the compilation. @gitoleg, can you please take a look into it?

JoranHonig commented 5 years ago

Nice. So first of all, the bap-objdump package is not very important, especially on a Mac OS X, so you can start playing with BAP right now.

Great news šŸ‘

For what it's worth, it doesn't seem like brew provided the objdump on my machine, looking at the output of objdump it seems like it indeed could be llvms' objdump:

>$ objdump
OVERVIEW: llvm object file dumper
ivg commented 5 years ago

looks like it worked finally :)

ivg commented 5 years ago

For those who stumble with the same problem, like that binutils do not work on Mac OS X, the suggestion is to just ignore that issue. This is a non-essential package and bap will work fine without it. Just continue using bap.


P.S. Logan Leland, I've got a notification from you on my email, but I can't reply there, as there is no return address of yours :) Also, if you have any problems it is better to create a new issue, or reopen existing, or just visit our gitter chat. It is hard in github to follow changes in closed issues.

loganleland commented 5 years ago

Thanks so much for the email,

I've successfully ignored the error message and have started writing my own plugins. So happy I've discovered BAP!


From: Ivan Gotovchits [notifications@github.com] Sent: Friday, May 3, 2019 10:54 AM To: BinaryAnalysisPlatform/bap Cc: Leland, Logan; Comment Subject: Re: [BinaryAnalysisPlatform/bap] Installation instructions Macos with Brew (#925)

For those who stumble with the same problem, like that binutils do not work on Mac OS X, the suggestion is to just ignore that issue. This is a non-essential package and bap will work fine without it. Just continue using bap.


P.S. Logan Leland, I've got a notification from you on my email, but I can't reply there, as there is no return address of yours :) Also, if you have any problems it is better to create a new issue, or reopen existing, or just visit our gitter chat. It is hard in github to follow changes in closed issues.

ā€” You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FBinaryAnalysisPlatform%2Fbap%2Fissues%2F925%23issuecomment-489123749&data=02%7C01%7Cll734713%40ohio.edu%7C06afc26b063c4370bc1d08d6cfd747e2%7Cf3308007477c4a70888934611817c55a%7C0%7C0%7C636924920863341448&sdata=ZMbbXmuVeXJrNT5IB0lT4IyBQx4P67LNQcyUNfCxXmo%3D&reserved=0, or mute the threadhttps://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABSQLRACHSVRPLYXGQMBUODPTRG3JANCNFSM4GYVAFWA&data=02%7C01%7Cll734713%40ohio.edu%7C06afc26b063c4370bc1d08d6cfd747e2%7Cf3308007477c4a70888934611817c55a%7C0%7C0%7C636924920863351460&sdata=jMnQcSn8UVqOR0gmj225RJ2dWFxuuZvQi3ZrT600ATg%3D&reserved=0.