search

google / bindiff

Quickly find differences and similarities in disassembled code
https://zynamics.com/bindiff.html
Apache License 2.0
2.15k stars 130 forks source link

Build instructions failing when building from source due to binaryninjaapi error #35

Closed nicolaipre closed 6 months ago

nicolaipre commented 6 months ago

Installing BinDiff from source according to building instructions in README currently fails due to binaryninjaapi git error.

Environment (please complete the following information):

Building from Source (From README.md):

The following build dependencies are optional:

IDA Pro only: IDA SDK 8.0 or higher (unpack into deps/idasdk) The general build steps are the same on Windows, Linux and macOS. The following shows the commands for Linux.

Download dependencies that won't be downloaded automatically:

mkdir -p build/out
git clone https://github.com/google/binexport build/binexport
unzip -q <path/to/idasdk_pro80.zip> -d build/idasdk

Next, configure the build directory and generate build files:

cmake -S . -B build/out -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=build/out \
  -DBINDIFF_BINEXPORT_DIR=build/binexport \
  "-DIdaSdk_ROOT_DIR=${PWD}build/idasdk"

Output error

user@laptop:~/bindiff-8$ cmake -S . -B build/out -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=build/out \
  -DBINDIFF_BINEXPORT_DIR=build/binexport \
  "-DIdaSdk_ROOT_DIR=${PWD}build/idasdk"
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test ABSL_INTERNAL_AT_LEAST_CXX17
-- Performing Test ABSL_INTERNAL_AT_LEAST_CXX17 - Success
-- Performing Test ABSL_INTERNAL_AT_LEAST_CXX20
-- Performing Test ABSL_INTERNAL_AT_LEAST_CXX20 - Failed
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- 
-- 25.2.0
-- Performing Test protobuf_HAVE_LD_VERSION_SCRIPT
-- Performing Test protobuf_HAVE_LD_VERSION_SCRIPT - Success
-- Performing Test protobuf_HAVE_BUILTIN_ATOMICS
-- Performing Test protobuf_HAVE_BUILTIN_ATOMICS - Success
-- Found Protobuf: protobuf::libprotobuf (found suitable version "4.25.2", minimum required is "3.14") 
[1/9] Creating directories for 'binaryninjaapi-populate'
[1/9] Performing download step (git clone) for 'binaryninjaapi-populate'
Cloning into 'binaryninjaapi-src'...
fatal: reference is not a tree: 0d57172be4fda0b7015db697b5353a12d03932af
CMake Error at binaryninjaapi-subbuild/binaryninjaapi-populate-prefix/tmp/binaryninjaapi-populate-gitclone.cmake:40 (message):
  Failed to checkout tag: '0d57172be4fda0b7015db697b5353a12d03932af'

FAILED: binaryninjaapi-populate-prefix/src/binaryninjaapi-populate-stamp/binaryninjaapi-populate-download /home/user/bindiff-8/build/out/_deps/binaryninjaapi-subbuild/binaryninjaapi-populate-prefix/src/binaryninjaapi-populate-stamp/binaryninjaapi-populate-download 
cd /home/user/bindiff-8/build/out/_deps && /usr/bin/cmake -P /home/user/bindiff-8/build/out/_deps/binaryninjaapi-subbuild/binaryninjaapi-populate-prefix/tmp/binaryninjaapi-populate-gitclone.cmake && /usr/bin/cmake -E touch /home/user/bindiff-8/build/out/_deps/binaryninjaapi-subbuild/binaryninjaapi-populate-prefix/src/binaryninjaapi-populate-stamp/binaryninjaapi-populate-download
ninja: build stopped: subcommand failed.

CMake Error at /usr/share/cmake-3.22/Modules/FetchContent.cmake:1087 (message):
  Build step for binaryninjaapi failed: 1
Call Stack (most recent call first):
  /usr/share/cmake-3.22/Modules/FetchContent.cmake:1216:EVAL:2 (__FetchContent_directPopulate)
  /usr/share/cmake-3.22/Modules/FetchContent.cmake:1216 (cmake_language)
  build/binexport/cmake/BinExportDeps.cmake:104 (FetchContent_Populate)
  build/binexport/CMakeLists.txt:42 (include)

-- Configuring incomplete, errors occurred!
See also "/home/user/bindiff-8/build/out/CMakeFiles/CMakeOutput.log".
See also "/home/user/bindiff-8/build/out/CMakeFiles/CMakeError.log".
cblichmann commented 6 months ago

This is odd - the referenced Binary Ninja commit exists, but GitHub seems to think that it is not part of the projects history.

Can you try to replace it with 59e569906828e91e4884670c2bba448702f5a31d in BinExportDeps.cmake?

nicolaipre commented 6 months ago

After that change it now builds successfully:

$ cmake -S . -B build/out -G Ninja   -DCMAKE_BUILD_TYPE=Release   -DCMAKE_INSTALL_PREFIX=build/out   -DBINDIFF_BINEXPORT_DIR=build/binexport   "-DIdaSdk_ROOT_DIR=${PWD}/build/idasdk/idasdk_pro84"
-- 
-- 25.2.0
-- Found Binary Ninja Core: binaryninjacore
-- Binary Ninja Link Dirs: 
-- Binary Ninja Install Dir: 
-- Binary Ninja User Plugins Dir: /home/user/.binaryninja/plugins
-- Found IdaSdk: /home/user/Downloads/tmp/bindiff-8/build/idasdk/idasdk_pro84  
-- Configuring done
-- Generating done
-- Build files have been written to: /home/user/Downloads/tmp/bindiff-8/build/out

Also, the idasdk_proNN.zip contains a folder with the same name, so the cmake command needs a minor change:

5c5
<   "-DIdaSdk_ROOT_DIR=${PWD}build/idasdk"
---
>   "-DIdaSdk_ROOT_DIR=${PWD}/build/idasdk/idasdk_pro84"

Or better yet just modify the unzip command accordingly.

Thank you for the help!

Edit: Seems like some of the tests are failing too (possibly because of the new unsupported update?):

$ (cd build/out; ctest --build-config Release --output-on-failure)

Test project /home/user/Downloads/tmp/bindiff-8/build/out
      Start  1: EmptyCallGraphDeathTest.QueryingVerticesCrashes
 1/25 Test  #1: EmptyCallGraphDeathTest.QueryingVerticesCrashes ........................   Passed    0.42 sec
      Start  2: EmptyCallGraphTest.Construction
 2/25 Test  #2: EmptyCallGraphTest.Construction ........................................   Passed    0.00 sec
      Start  3: EmptyCallGraphTest.AddOrRemoveNullFlowGraphThrows
 3/25 Test  #3: EmptyCallGraphTest.AddOrRemoveNullFlowGraphThrows ......................   Passed    0.00 sec
      Start  4: EmptyCallGraphTest.CrossPlatformFileBasenames
 4/25 Test  #4: EmptyCallGraphTest.CrossPlatformFileBasenames ..........................   Passed    0.00 sec
      Start  5: SimpleCallGraphTest.Construction
 5/25 Test  #5: SimpleCallGraphTest.Construction .......................................   Passed    0.00 sec
      Start  6: SimpleCallGraphTest.TopologyValid
 6/25 Test  #6: SimpleCallGraphTest.TopologyValid ......................................   Passed    0.00 sec
      Start  7: ChangeClassifierTest.ChangeDescription
 7/25 Test  #7: ChangeClassifierTest.ChangeDescription .................................   Passed    0.01 sec
      Start  8: ChangeClassifierTest.BasicChange
 8/25 Test  #8: ChangeClassifierTest.BasicChange .......................................   Passed    0.01 sec
      Start  9: DatabaseWriterTest.SimpleDatabaseCreation
 9/25 Test  #9: DatabaseWriterTest.SimpleDatabaseCreation ..............................   Passed    0.05 sec
      Start 10: FlowGraphMatchBasicBlockInstructionCountTest.DefaultFlowGraphMatches
10/25 Test #10: FlowGraphMatchBasicBlockInstructionCountTest.DefaultFlowGraphMatches ...   Passed    0.01 sec
      Start 11: PrimeSignatureTest.IPow32MathEdgeCases
11/25 Test #11: PrimeSignatureTest.IPow32MathEdgeCases .................................   Passed    0.00 sec
      Start 12: PrimeSignatureTest.IPow32NonOverflow
12/25 Test #12: PrimeSignatureTest.IPow32NonOverflow ...................................   Passed    0.00 sec
      Start 13: PrimeSignatureTest.IPow32Overflow
13/25 Test #13: PrimeSignatureTest.IPow32Overflow ......................................   Passed    0.00 sec
      Start 14: PrimeSignatureTest.GetPrimeDistinctX86Mnemonics
14/25 Test #14: PrimeSignatureTest.GetPrimeDistinctX86Mnemonics ........................   Passed    0.00 sec
      Start 15: PrimeSignatureTest.GetPrimeCheckCollision
15/25 Test #15: PrimeSignatureTest.GetPrimeCheckCollision ..............................   Passed    0.00 sec
      Start 16: SqliteTest.BasicFunctionality
16/25 Test #16: SqliteTest.BasicFunctionality ..........................................   Passed    0.01 sec
      Start 17: SqliteTest.InvalidSyntax
17/25 Test #17: SqliteTest.InvalidSyntax ...............................................   Passed    0.00 sec
      Start 18: SqliteTest.RollbackTransaction
18/25 Test #18: SqliteTest.RollbackTransaction .........................................   Passed    0.01 sec
      Start 19: LogWriterTest.Empty
19/25 Test #19: LogWriterTest.Empty ....................................................   Passed    0.02 sec
      Start 20: WriterTest.EmptyChainDoesNothing
20/25 Test #20: WriterTest.EmptyChainDoesNothing .......................................   Passed    0.01 sec
      Start 21: WriterTest.CanChainWriters
21/25 Test #21: WriterTest.CanChainWriters .............................................   Passed    0.01 sec
      Start 22: binexport_shared_test_NOT_BUILT
Could not find executable binexport_shared_test_NOT_BUILT
Looked in the following places:
binexport_shared_test_NOT_BUILT
binexport_shared_test_NOT_BUILT
Release/binexport_shared_test_NOT_BUILT
Release/binexport_shared_test_NOT_BUILT
Release/binexport_shared_test_NOT_BUILT
Release/binexport_shared_test_NOT_BUILT
Unable to find executable: binexport_shared_test_NOT_BUILT
22/25 Test #22: binexport_shared_test_NOT_BUILT ........................................***Not Run   0.00 sec
      Start 23: idb_export_test_NOT_BUILT
Could not find executable idb_export_test_NOT_BUILT
Looked in the following places:
idb_export_test_NOT_BUILT
idb_export_test_NOT_BUILT
Release/idb_export_test_NOT_BUILT
Release/idb_export_test_NOT_BUILT
Release/idb_export_test_NOT_BUILT
Release/idb_export_test_NOT_BUILT
Unable to find executable: idb_export_test_NOT_BUILT
23/25 Test #23: idb_export_test_NOT_BUILT ..............................................***Not Run   0.00 sec
      Start 24: binexport_test_NOT_BUILT
Could not find executable binexport_test_NOT_BUILT
Looked in the following places:
binexport_test_NOT_BUILT
binexport_test_NOT_BUILT
Release/binexport_test_NOT_BUILT
Release/binexport_test_NOT_BUILT
Release/binexport_test_NOT_BUILT
Release/binexport_test_NOT_BUILT
Unable to find executable: binexport_test_NOT_BUILT
24/25 Test #24: binexport_test_NOT_BUILT ...............................................***Not Run   0.00 sec
      Start 25: binexport_reader_test_NOT_BUILT
Could not find executable binexport_reader_test_NOT_BUILT
Looked in the following places:
binexport_reader_test_NOT_BUILT
binexport_reader_test_NOT_BUILT
Release/binexport_reader_test_NOT_BUILT
Release/binexport_reader_test_NOT_BUILT
Release/binexport_reader_test_NOT_BUILT
Release/binexport_reader_test_NOT_BUILT
Unable to find executable: binexport_reader_test_NOT_BUILT
25/25 Test #25: binexport_reader_test_NOT_BUILT ........................................***Not Run   0.00 sec

84% tests passed, 4 tests failed out of 25

Total Test time (real) =   0.59 sec

The following tests FAILED:
     22 - binexport_shared_test_NOT_BUILT (Not Run)
     23 - idb_export_test_NOT_BUILT (Not Run)
     24 - binexport_test_NOT_BUILT (Not Run)
     25 - binexport_reader_test_NOT_BUILT (Not Run)
Errors while running CTest
cblichmann commented 6 months ago

After that change it now builds successfully

Glad to hear.

Also, the idasdk_proNN.zip contains a folder with the same name, so the cmake command needs a minor change

Note that the directory names inside the ZIP are different for IDA Home, IDA Pro and IDA Teams, which is rather annoying. I should probably change the command to include a shell glob (idaask_*.zip).

Seems like some of the tests are failing too (possibly because of the new unsupported update?)

This is expected. The failing tests are all BinExport tests that live in the BinExport repo and are not built by default. Somehow, ctest is still picking them up as tests to be run.

nicolaipre commented 6 months ago

Understandable, and thank you for the information.

Bindiff built successfully. I also built Binexport and had to apply the same changes to the BinExportDeps.cmake file there, but after that everything works.

Feel free to close the issue.