Problem Compiling BlockSci on Mac OS

jptbrandao commented 4 years ago

Hello, I'm following the instructions in https://citp.github.io/BlockSci/compiling.html to compile BlockSci locally, and I'm getting errors after running the command 'make'.

Reproduction Steps

brew install cmake jsoncpp libjson-rpc-cpp boost openssl jemalloc zstd \
automake libtool google-sparsehash python3
sudo xcode-select --reset

git clone https://github.com/citp/BlockSci.git
cd BlockSci
mkdir release
cd release
cmake -DCMAKE_BUILD_TYPE=Release -DOPENSSL_ROOT_DIR=/usr/local/opt/openssl ..
make

Output

...
[  2%] Completed 'project_secp256k1'
[  2%] Built target project_secp256k1
Scanning dependencies of target copy_externals
[  2%] Built target copy_externals
Scanning dependencies of target build_version
[  2%] Building CXX object external/rocksdb/CMakeFiles/build_version.dir/__/__/build_version.cc.o
[  2%] Built target build_version
Scanning dependencies of target rocksdb
[  2%] Building CXX object external/rocksdb/CMakeFiles/rocksdb.dir/cache/clock_cache.cc.o
[  2%] Building CXX object external/rocksdb/CMakeFiles/rocksdb.dir/cache/lru_cache.cc.o
[  2%] Building CXX object external/rocksdb/CMakeFiles/rocksdb.dir/cache/sharded_cache.cc.o
[  2%] Building CXX object external/rocksdb/CMakeFiles/rocksdb.dir/db/builder.cc.o
In file included from /Users/brandao/projects/github/BlockSci/external/rocksdb/db/builder.cc:19:
/Users/brandao/projects/github/BlockSci/external/rocksdb/db/internal_stats.h:107:5: error: declaration shadows a variable in namespace 'rocksdb' [-Werror,-Wshadow]
    WAL_FILE_BYTES,
    ^
/Users/brandao/projects/github/BlockSci/external/rocksdb/include/rocksdb/statistics.h:191:3: note: previous declaration is here
  WAL_FILE_BYTES,   // Number of bytes written to WAL
  ^
In file included from /Users/brandao/projects/github/BlockSci/external/rocksdb/db/builder.cc:19:
/Users/brandao/projects/github/BlockSci/external/rocksdb/db/internal_stats.h:108:5: error: declaration shadows a variable in namespace 'rocksdb' [-Werror,-Wshadow]
    WAL_FILE_SYNCED,
    ^
/Users/brandao/projects/github/BlockSci/external/rocksdb/include/rocksdb/statistics.h:190:3: note: previous declaration is here
  WAL_FILE_SYNCED,  // Number of times WAL sync is done
  ^
In file included from /Users/brandao/projects/github/BlockSci/external/rocksdb/db/builder.cc:19:
/Users/brandao/projects/github/BlockSci/external/rocksdb/db/internal_stats.h:109:5: error: declaration shadows a variable in namespace 'rocksdb' [-Werror,-Wshadow]
    BYTES_WRITTEN,
    ^
/Users/brandao/projects/github/BlockSci/external/rocksdb/include/rocksdb/statistics.h:127:3: note: previous declaration is here
  BYTES_WRITTEN,
  ^
In file included from /Users/brandao/projects/github/BlockSci/external/rocksdb/db/builder.cc:19:
/Users/brandao/projects/github/BlockSci/external/rocksdb/db/internal_stats.h:110:5: error: declaration shadows a variable in namespace 'rocksdb' [-Werror,-Wshadow]
    NUMBER_KEYS_WRITTEN,
    ^
/Users/brandao/projects/github/BlockSci/external/rocksdb/include/rocksdb/statistics.h:120:3: note: previous declaration is here
  NUMBER_KEYS_WRITTEN,
  ^
In file included from /Users/brandao/projects/github/BlockSci/external/rocksdb/db/builder.cc:19:
/Users/brandao/projects/github/BlockSci/external/rocksdb/db/internal_stats.h:111:5: error: declaration shadows a variable in namespace 'rocksdb' [-Werror,-Wshadow]
    WRITE_DONE_BY_OTHER,
    ^
/Users/brandao/projects/github/BlockSci/external/rocksdb/include/rocksdb/statistics.h:196:3: note: previous declaration is here
  WRITE_DONE_BY_OTHER,  // Equivalent to writes done for others
  ^
In file included from /Users/brandao/projects/github/BlockSci/external/rocksdb/db/builder.cc:19:
/Users/brandao/projects/github/BlockSci/external/rocksdb/db/internal_stats.h:112:5: error: declaration shadows a variable in namespace 'rocksdb' [-Werror,-Wshadow]
    WRITE_DONE_BY_SELF,
    ^
/Users/brandao/projects/github/BlockSci/external/rocksdb/include/rocksdb/statistics.h:195:3: note: previous declaration is here
  WRITE_DONE_BY_SELF,
  ^
In file included from /Users/brandao/projects/github/BlockSci/external/rocksdb/db/builder.cc:19:
/Users/brandao/projects/github/BlockSci/external/rocksdb/db/internal_stats.h:113:5: error: declaration shadows a variable in namespace 'rocksdb' [-Werror,-Wshadow]
    WRITE_WITH_WAL,
    ^
/Users/brandao/projects/github/BlockSci/external/rocksdb/include/rocksdb/statistics.h:198:3: note: previous declaration is here
  WRITE_WITH_WAL,       // Number of Write calls that request WAL
  ^
7 errors generated.
make[2]: *** [external/rocksdb/CMakeFiles/rocksdb.dir/db/builder.cc.o] Error 1
make[1]: *** [external/rocksdb/CMakeFiles/rocksdb.dir/all] Error 2
make: *** [all] Error 2

System Information

Mac OS Catalina v10.15.4 Total memory: 32 GB

maltemoeser commented 4 years ago

Try using the v0.6 branch (git checkout v0.6), and you might need to locally merge #386

jptbrandao commented 4 years ago

Thanks for the tip.

I did as you suggested, but had to also manually install lz4 libraries, before running the commando

cmake -DCMAKE_BUILD_TYPE=Release -DOPENSSL_ROOT_DIR=/usr/local/opt/openssl ..

When running the 'make' command afterwards I get the following errors: clang: error: linker command failed with exit code 1 (use -v to see invocation)

Here is the last part of the output:

/Users/brandao/projects/github/BlockSci/src/../include/blocksci/external/concepts/concepts.hpp:1178:13: note: declare 'static' if the variable is not intended to be used outside of this translation unit
            CPP_concept convertible_to =
            ^
/Users/brandao/projects/github/BlockSci/src/../include/blocksci/external/concepts/concepts.hpp:226:46: note: expanded from macro 'CPP_concept'
#define CPP_concept CPP_INLINE_VAR constexpr auto
                                             ^
699 warnings generated.
[ 92%] Linking CXX shared library libblocksci.dylib
duplicate symbol 'ranges::detail::_is_integer_like_<ranges::detail::diffmax_t>' in:
    CMakeFiles/blocksci.dir/scripts/script_range.cpp.o
    CMakeFiles/blocksci.dir/chain/transaction.cpp.o
duplicate symbol 'ranges::detail::_is_integer_like_<ranges::detail::diffmax_t>' in:
    CMakeFiles/blocksci.dir/scripts/script_range.cpp.o
    CMakeFiles/blocksci.dir/heuristics/change_address.cpp.o
duplicate symbol 'ranges::detail::_is_integer_like_<ranges::detail::diffmax_t>' in:
    CMakeFiles/blocksci.dir/scripts/script_range.cpp.o
    CMakeFiles/blocksci.dir/cluster/cluster_manager.cpp.o
ld: 3 duplicate symbols for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [src/libblocksci.dylib] Error 1
make[1]: *** [src/CMakeFiles/blocksci.dir/all] Error 2
make: *** [all] Error 2

maltemoeser commented 4 years ago

Can confirm the error after upgrading to Xcode 11.5. I'll look into it.

jptbrandao commented 4 years ago

Thanks

maltemoeser commented 4 years ago

Seems like this particular issue was fixed in https://github.com/ericniebler/range-v3/issues/1441, but after cherry-picking this commit there are other errors. Ideally, I'd want to wait for a new release of range-v3 instead of patching together something (though I might give it a try if I find the time).

maltemoeser commented 4 years ago

Here is an experimental fix (I won't maintain this branch) to get BlockSci compiling on MacOS with the latest XCode. This has the RocksDB upgrade merged, so it's not compatible with old parsings:

git fetch --all
git checkout macos-build-fix
git submodule sync
git submodule update --recursive
cd external/range-v3
git fetch --all
git cherry-pick ff648328dacd097f1fa3abd1cfee4dc611e5af98 -x
cd ../../release
make

jptbrandao commented 4 years ago

Thanks, are you sure about the 3rd to last line?

The output gives me: 'fatal: bad object 'ff648328dacd097f1fa3abd1cfee4dc611e5af98'

maltemoeser commented 4 years ago

Try a git fetch --all in the external/range-v3 folder

jptbrandao commented 4 years ago

That worked! Thanks!

The full commands are: git fetch --all git checkout macos-build-fix git submodule sync git submodule update --recursive cd external/range-v3 git fetch --all git cherry-pick ff648328dacd097f1fa3abd1cfee4dc611e5af98 -x cd ../../release make

maltemoeser commented 4 years ago

Okay perfect! I've updated the instructions above and will keep this open until it has been resolved outside of that patch branch.

citp / BlockSci