search

mit-dci / opencbdc-tx

A transaction processor for a hypothetical, general-purpose, central bank digital currency
Other
901 stars 199 forks source link

Build fails on MacOS #36

Closed dmtrinh closed 2 years ago

dmtrinh commented 2 years ago

Affected Branch

trunk

Basic Diagnostics

Description

Trying to build this directly after cloning opencbdc-tx repository. Linking failed with:

Undefined symbols for architecture x86_64:
  "typeinfo for leveldb::Comparator", referenced from:
      typeinfo for cbdc::raft::index_comparator in libraft.a(index_comparator.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Environment

(base) ---------- 20:24:48 (duc@DucMacBook) :~/Dev2/opencbdc-tx ----------
==> clang --version
Apple clang version 12.0.5 (clang-1205.0.22.11)
Target: x86_64-apple-darwin20.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
(base) ---------- 20:25:25 (duc@DucMacBook) :~/Dev2/opencbdc-tx ----------
==> system_profiler SPHardwareDataType
Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: MacBookPro15,1
      Processor Name: 6-Core Intel Core i7
      Processor Speed: 2.6 GHz
      Number of Processors: 1
      Total Number of Cores: 6
      L2 Cache (per Core): 256 KB
      L3 Cache: 9 MB
      Hyper-Threading Technology: Enabled
      Memory: 16 GB
      System Firmware Version: 1554.140.20.0.0 (iBridge: 18.16.14759.0.1,0)
      Activation Lock Status: Disabled

To Reproduce

  1. git clone --recurse-submodules https://github.com/mit-dci/opencbdc-tx
  2. Change into opencbdc-tx directory and run scripts/configure.sh 
    
(base) ---------- 20:14:06 (duc@DucMacBook) :~/Dev2/opencbdc-tx ----------
==> scripts/configure.sh
Configuring...
Warning: leveldb 1.23 is already installed and up-to-date.
To reinstall 1.23, run:
brew reinstall leveldb
Warning: llvm@11 11.1.0_4 is already installed and up-to-date.
To reinstall 11.1.0_4, run:
brew reinstall llvm@11
Warning: googletest 1.11.0 is already installed and up-to-date.
To reinstall 1.11.0, run:
brew reinstall googletest
Warning: lcov 1.15 is already installed and up-to-date.
To reinstall 1.15, run:
brew reinstall lcov
Warning: make 4.3 is already installed and up-to-date.
To reinstall 4.3, run:
brew reinstall make
To run clang-tidy, you must add it to your path. Ex: ln -s /usr/local/opt/llvm@11/bin/clang-tidy /usr/local/bin/clang-tidy
Installing NuRaft from cache...
--2022-02-06 20:14:16--  https://raw.githubusercontent.com/llvm/llvm-project/e837ce2a32369b2e9e8e5d60270c072c7dd63827/clang-tools-extra/clang-tidy/tool/run-clang-tidy.py
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.108.133, 185.199.110.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 12311 (12K) [text/plain]
Saving to: ‘/usr/local/bin/run-clang-tidy.py.5’

run-clang-tidy.py.5 100%[=============================================================>] 12.02K --.-KB/s in 0s

2022-02-06 20:14:16 (31.4 MB/s) - ‘/usr/local/bin/run-clang-tidy.py.5’ saved [12311/12311]

3.  Run `scripts/builld.sh`

(base) ---------- 20:14:16 (duc@DucMacBook) :~/Dev2/opencbdc-tx ---------- ==> scripts/build.sh
Building... -- Configuring done -- Generating done -- Build files have been written to: /Users/duc/Dev2/opencbdc-tx/build Consolidate compiler generated dependencies of target secp256k1_genctx Consolidate compiler generated dependencies of target bech32 Consolidate compiler generated dependencies of target archiver Consolidate compiler generated dependencies of target crypto Consolidate compiler generated dependencies of target serialization Consolidate compiler generated dependencies of target transaction Consolidate compiler generated dependencies of target atomizer_raft Consolidate compiler generated dependencies of target atomizer Consolidate compiler generated dependencies of target network Consolidate compiler generated dependencies of target common Consolidate compiler generated dependencies of target watchtower Consolidate compiler generated dependencies of target raft [ 1%] Building C object 3rdparty/CMakeFiles/secp256k1_genctx.dir/secp256k1/src/gen_context.c.o [ 1%] Building CXX object 3rdparty/bech32/CMakeFiles/bech32.dir/bech32.o [ 3%] Building CXX object 3rdparty/crypto/CMakeFiles/crypto.dir/sha256_avx2.o [ 3%] Building CXX object src/uhs/atomizer/atomizer/CMakeFiles/atomizer_raft.dir/atomizer_raft.o [ 3%] Building CXX object src/uhs/atomizer/archiver/CMakeFiles/archiver.dir/client.o [ 4%] Building CXX object src/util/serialization/CMakeFiles/serialization.dir/format.o [ 5%] Building CXX object src/uhs/transaction/CMakeFiles/transaction.dir/transaction.o [ 7%] Building CXX object src/util/raft/CMakeFiles/raft.dir/console_logger.o [ 7%] Building CXX object src/util/common/CMakeFiles/common.dir/buffer.o [ 8%] Building CXX object src/uhs/atomizer/atomizer/CMakeFiles/atomizer.dir/atomizer.o [ 9%] Building CXX object src/util/network/CMakeFiles/network.dir/connection_manager.o [ 9%] Building CXX object src/uhs/atomizer/watchtower/CMakeFiles/watchtower.dir/controller.o [ 9%] Linking C executable secp256k1_genctx [ 9%] Built target secp256k1_genctx [ 10%] Building CXX object src/uhs/atomizer/archiver/CMakeFiles/archiver.dir/controller.o [ 10%] Building CXX object 3rdparty/crypto/CMakeFiles/crypto.dir/sha256_shani.o [ 11%] Linking CXX static library libbech32.a [ 11%] Built target bech32 [ 11%] Building CXX object src/util/serialization/CMakeFiles/serialization.dir/buffer_serializer.o [ 11%] Building CXX object src/util/common/CMakeFiles/common.dir/hash.o [ 12%] Building CXX object 3rdparty/crypto/CMakeFiles/crypto.dir/sha256_sse4.o [ 12%] Building CXX object src/uhs/atomizer/atomizer/CMakeFiles/atomizer_raft.dir/controller.o [ 12%] Building CXX object 3rdparty/crypto/CMakeFiles/crypto.dir/sha256_sse41.o [ 13%] Building CXX object src/util/serialization/CMakeFiles/serialization.dir/size_serializer.o [ 13%] Building CXX object src/util/serialization/CMakeFiles/serialization.dir/stream_serializer.o [ 13%] Building CXX object src/uhs/transaction/CMakeFiles/transaction.dir/messages.o [ 13%] Building CXX object src/util/raft/CMakeFiles/raft.dir/state_manager.o [ 13%] Building CXX object src/util/network/CMakeFiles/network.dir/peer.o [ 14%] Building CXX object 3rdparty/crypto/CMakeFiles/crypto.dir/sha256.o [ 15%] Building CXX object src/util/common/CMakeFiles/common.dir/hashmap.o [ 16%] Building CXX object src/util/serialization/CMakeFiles/serialization.dir/istream_serializer.o [ 17%] Building CXX object src/uhs/atomizer/watchtower/CMakeFiles/watchtower.dir/watchtower.o [ 18%] Building CXX object 3rdparty/crypto/CMakeFiles/crypto.dir/siphash.o [ 18%] Linking CXX static library libcrypto.a [ 19%] Building CXX object src/util/common/CMakeFiles/common.dir/keys.o [ 19%] Built target crypto [ 20%] Building CXX object src/uhs/transaction/CMakeFiles/transaction.dir/validation.o [ 21%] Building CXX object src/uhs/atomizer/atomizer/CMakeFiles/atomizer.dir/block.o [ 22%] Building CXX object src/util/serialization/CMakeFiles/serialization.dir/ostream_serializer.o [ 23%] Building CXX object src/util/raft/CMakeFiles/raft.dir/log_store.o [ 24%] Building CXX object src/util/network/CMakeFiles/network.dir/socket.o [ 24%] Building CXX object src/util/common/CMakeFiles/common.dir/config.o [ 24%] Building CXX object src/uhs/atomizer/watchtower/CMakeFiles/watchtower.dir/status_update.o [ 25%] Building CXX object src/uhs/atomizer/atomizer/CMakeFiles/atomizer_raft.dir/state_machine.o Consolidate compiler generated dependencies of target sentinel [ 25%] Linking CXX static library libserialization.a [ 26%] Building CXX object src/uhs/atomizer/sentinel/CMakeFiles/sentinel.dir/controller.o [ 26%] Building CXX object src/util/network/CMakeFiles/network.dir/socket_selector.o [ 26%] Built target serialization [ 27%] Building CXX object src/uhs/atomizer/watchtower/CMakeFiles/watchtower.dir/error_cache.o [ 27%] Linking CXX static library libarchiver.a [ 27%] Built target archiver [ 28%] Building CXX object src/util/raft/CMakeFiles/raft.dir/node.o [ 29%] Building CXX object src/uhs/transaction/CMakeFiles/transaction.dir/wallet.o [ 29%] Building CXX object src/uhs/atomizer/atomizer/CMakeFiles/atomizer.dir/state_machine.o [ 30%] Building CXX object src/util/network/CMakeFiles/network.dir/tcp_listener.o [ 30%] Building CXX object src/uhs/atomizer/watchtower/CMakeFiles/watchtower.dir/block_cache.o [ 31%] Building CXX object src/util/common/CMakeFiles/common.dir/logging.o [ 32%] Building CXX object src/util/network/CMakeFiles/network.dir/tcp_socket.o [ 33%] Building CXX object src/uhs/atomizer/atomizer/CMakeFiles/atomizer.dir/format.o [ 33%] Linking CXX static library libnetwork.a [ 33%] Built target network [ 33%] Building CXX object src/uhs/atomizer/sentinel/CMakeFiles/sentinel.dir/server.o [ 33%] Building CXX object src/util/raft/CMakeFiles/raft.dir/serialization.o [ 34%] Building CXX object src/uhs/atomizer/watchtower/CMakeFiles/watchtower.dir/tx_error_messages.o [ 34%] Building CXX object src/util/common/CMakeFiles/common.dir/random_source.o Consolidate compiler generated dependencies of target sentinel_interface [ 34%] Building CXX object src/uhs/sentinel/CMakeFiles/sentinel_interface.dir/format.o [ 35%] Building CXX object src/uhs/atomizer/watchtower/CMakeFiles/watchtower.dir/status_update_messages.o [ 36%] Building CXX object src/util/raft/CMakeFiles/raft.dir/messages.o [ 36%] Building CXX object src/uhs/atomizer/atomizer/CMakeFiles/atomizer.dir/messages.o [ 37%] Building CXX object src/uhs/sentinel/CMakeFiles/sentinel_interface.dir/client.o [ 37%] Linking CXX static library libtransaction.a [ 37%] Built target transaction [ 37%] Building CXX object src/uhs/atomizer/watchtower/CMakeFiles/watchtower.dir/messages.o [ 37%] Building CXX object src/util/raft/CMakeFiles/raft.dir/index_comparator.o [ 38%] Linking CXX static library libatomizer_raft.a [ 39%] Linking CXX static library libcommon.a [ 39%] Built target atomizer_raft Consolidate compiler generated dependencies of target rpc [ 39%] Built target common [ 40%] Building CXX object src/uhs/atomizer/watchtower/CMakeFiles/watchtower.dir/client.o [ 40%] Building CXX object src/uhs/sentinel/CMakeFiles/sentinel_interface.dir/interface.o [ 40%] Building CXX object src/util/rpc/CMakeFiles/rpc.dir/format.o [ 41%] Linking CXX static library libsentinel.a [ 42%] Linking CXX static library libraft.a [ 42%] Built target sentinel Consolidate compiler generated dependencies of target shard [ 43%] Building CXX object src/uhs/atomizer/shard/CMakeFiles/shard.dir/shard.o [ 43%] Built target raft Consolidate compiler generated dependencies of target client Consolidate compiler generated dependencies of target coordinator [ 43%] Building CXX object src/uhs/client/CMakeFiles/client.dir/atomizer_client.o [ 43%] Building CXX object src/uhs/twophase/coordinator/CMakeFiles/coordinator.dir/format.o [ 44%] Building CXX object src/uhs/client/CMakeFiles/client.dir/client.o [ 45%] Linking CXX static library libatomizer.a [ 45%] Built target atomizer Consolidate compiler generated dependencies of target locking_shard [ 46%] Building CXX object src/uhs/twophase/locking_shard/CMakeFiles/locking_shard.dir/client.o [ 46%] Building CXX object src/uhs/atomizer/shard/CMakeFiles/shard.dir/controller.o Consolidate compiler generated dependencies of target sentinel_2pc [ 46%] Building CXX object src/uhs/twophase/sentinel_2pc/CMakeFiles/sentinel_2pc.dir/controller.o [ 46%] Building CXX object src/uhs/client/CMakeFiles/client.dir/twophase_client.o [ 47%] Linking CXX static library librpc.a [ 47%] Built target rpc [ 48%] Building CXX object src/uhs/twophase/coordinator/CMakeFiles/coordinator.dir/state_machine.o [ 48%] Building CXX object src/uhs/twophase/locking_shard/CMakeFiles/locking_shard.dir/controller.o [ 49%] Linking CXX static library libsentinel_interface.a [ 49%] Built target sentinel_interface [ 50%] Building CXX object src/uhs/twophase/sentinel_2pc/CMakeFiles/sentinel_2pc.dir/server.o [ 51%] Building CXX object src/uhs/twophase/locking_shard/CMakeFiles/locking_shard.dir/locking_shard.o [ 51%] Linking CXX static library libwatchtower.a [ 51%] Built target watchtower Consolidate compiler generated dependencies of target util [ 52%] Building CXX object src/uhs/twophase/coordinator/CMakeFiles/coordinator.dir/client.o [ 53%] Building CXX object tests/CMakeFiles/util.dir/util.o [ 54%] Building CXX object src/uhs/twophase/locking_shard/CMakeFiles/locking_shard.dir/interface.o [ 54%] Building CXX object src/uhs/twophase/coordinator/CMakeFiles/coordinator.dir/distributed_tx.o [ 55%] Building CXX object src/uhs/twophase/coordinator/CMakeFiles/coordinator.dir/controller.o [ 56%] Linking CXX static library libshard.a [ 56%] Built target shard [ 56%] Building CXX object src/uhs/twophase/locking_shard/CMakeFiles/locking_shard.dir/format.o [ 56%] Building CXX object src/uhs/twophase/coordinator/CMakeFiles/coordinator.dir/server.o [ 57%] Built target secp256k1_genctx_run [ 58%] Linking CXX static library libclient.a [ 59%] Building CXX object src/uhs/twophase/locking_shard/CMakeFiles/locking_shard.dir/messages.o [ 59%] Building CXX object src/uhs/twophase/locking_shard/CMakeFiles/locking_shard.dir/state_machine.o [ 60%] Linking CXX static library libsentinel_2pc.a [ 60%] Built target client [ 61%] Building CXX object src/uhs/twophase/locking_shard/CMakeFiles/locking_shard.dir/status_client.o [ 61%] Built target sentinel_2pc Consolidate compiler generated dependencies of target secp256k1 [ 62%] Building C object 3rdparty/CMakeFiles/secp256k1.dir/secp256k1/src/secp256k1.c.o [ 63%] Building CXX object src/uhs/twophase/locking_shard/CMakeFiles/locking_shard.dir/status_server.o [ 63%] Linking C static library libsecp256k1.a [ 63%] Built target secp256k1 Consolidate compiler generated dependencies of target archiverd Consolidate compiler generated dependencies of target atomizer-raftd Consolidate compiler generated dependencies of target sentineld [ 65%] Building CXX object src/uhs/atomizer/archiver/CMakeFiles/archiverd.dir/archiverd.o [ 65%] Building CXX object src/uhs/atomizer/atomizer/CMakeFiles/atomizer-raftd.dir/atomizer-raftd.o [ 65%] Building CXX object src/uhs/atomizer/sentinel/CMakeFiles/sentineld.dir/sentineld.o Consolidate compiler generated dependencies of target shardd [ 66%] Building CXX object src/uhs/atomizer/shard/CMakeFiles/shardd.dir/shardd.o [ 67%] Linking CXX static library libutil.a Consolidate compiler generated dependencies of target watchtowerd [ 67%] Built target util [ 68%] Building CXX object src/uhs/atomizer/watchtower/CMakeFiles/watchtowerd.dir/watchtowerd.o Consolidate compiler generated dependencies of target atomizer-cli-watchtower [ 68%] Building CXX object tools/bench/CMakeFiles/atomizer-cli-watchtower.dir/atomizer-cli-watchtower.o Consolidate compiler generated dependencies of target shard-seeder [ 69%] Building CXX object tools/shard-seeder/CMakeFiles/shard-seeder.dir/shard-seeder.o [ 69%] Linking CXX executable archiverd [ 70%] Linking CXX executable sentineld [ 70%] Linking CXX static library liblocking_shard.a [ 70%] Built target sentineld [ 70%] Built target locking_shard [ 70%] Built target archiverd Consolidate compiler generated dependencies of target locking-shardd [ 71%] Building CXX object src/uhs/twophase/locking_shard/CMakeFiles/locking-shardd.dir/locking_shardd.o [ 72%] Linking CXX executable watchtowerd [ 72%] Linking CXX executable shardd [ 72%] Linking CXX executable atomizer-raftd [ 72%] Linking CXX executable shard-seeder Undefined symbols for architecture x86_64: "typeinfo for leveldb::Comparator", referenced from: typeinfo for cbdc::raft::index_comparator in libraft.a(index_comparator.o) ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: [src/uhs/atomizer/atomizer/atomizer-raftd] Error 1 make[1]: [src/uhs/atomizer/atomizer/CMakeFiles/atomizer-raftd.dir/all] Error 2 make[1]: Waiting for unfinished jobs.... [ 72%] Built target shardd [ 72%] Built target watchtowerd [ 72%] Built target shard-seeder [ 73%] Linking CXX static library libcoordinator.a [ 73%] Built target coordinator [ 73%] Linking CXX executable locking-shardd Undefined symbols for architecture x86_64: "typeinfo for leveldb::Comparator", referenced from: typeinfo for cbdc::raft::index_comparator in libraft.a(index_comparator.o) ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: [src/uhs/twophase/locking_shard/locking-shardd] Error 1 make[1]: [src/uhs/twophase/locking_shard/CMakeFiles/locking-shardd.dir/all] Error 2 [ 74%] Linking CXX executable atomizer-cli-watchtower [ 74%] Built target atomizer-cli-watchtower make: [all] Error 2 (base) ---------- 20:14:49 (duc@DucMacBook) :~/Dev2/opencbdc-tx ---------- ==> 



### Code of Conduct

- [X] I agree to follow this project's Code of Conduct
HalosGhost commented 2 years ago

Thank you for making such a detailed bug report! I don't have a mac machine immediately at-hand, but we are working to reproduce this build error so we can offer a fix!

HalosGhost commented 2 years ago

In talking with a few folks who do have Apple hardware, it sounds like LevelDB may need to be compiled with RTTI disabled. In particular, if you apply the following patch to the leveldb source code before the compile/install step in ./scripts/configure.sh, it should resolve this error:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 7ecf317..548108c 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -72,8 +72,8 @@ else(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
   set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-exceptions")

   # Disable RTTI.
-  string(REGEX REPLACE "-frtti" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
-  set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-rtti")
+  #string(REGEX REPLACE "-frtti" "" CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS}")
+  #set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-rtti")
 endif(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")

 # Test whether -Wthread-safety is available. See
HalosGhost commented 2 years ago

I'm going to go ahead and close this for now; however, if you know of a way this can be easily fixed without needing the upstream change, we would happily accept a PR!

(To that end, I'm going to mark it as needing more information (which, in this case, would be the fix itself) rather than wont-fix.)

theuni commented 2 years ago

Ok, I've tracked down the cause of this.

Upstream leveldb has now added -fno-exceptions and -fno-rtti to its build by default. This is not the case for the older version we fetch in the configure script.

So afaik this would happen on Linux too when using the latest leveldb.

They're strange (read: harmful, IMO, since those are kinda compiler hacks which affect the abi) flags for a library to set, and it's even weirder that they can't be turned off. Upstream has a bunch of bug reports about the issue. I suspect they'll end up reverting it eventually.

So the options are:

  1. Require the old leveldb.
  2. Build all files which pull in leveldb typeinfo (I believe it's just those with classes which inherit from leveldb) with -fno-rtti to avoid emitting the unresolved symbols.
  3. Build all files with -fno-rtti
  4. Build old leveldb as part of our buildsystem
  5. Build new leveldb as part of the buildsystem and patch out the rtti if not also 2. or 3.

As an immediate fix, I suggest doing 3., since we already have exceptions disabled for pretty much the same reason. There's only a single use of dynamic_cast in the codebase which looks like it should be a reinterpret_cast anyway.

As a follow-up, I'd suggest 4 or 5, using as a tie-breaker whatever Bitcoin Core's current version is, since it does a good job of exposing bugs.

HalosGhost commented 2 years ago

I think I'm mostly in the camp of options 3–5. Thank you for investigating @theuni! I'm reopening this issue for us to reference in PRs which address the problem!