Compilation failure for v1.6.0 on macOS

[p-linnane]

Hello 👋 . I'm a maintainer for the Homebrew project. abyss has started using btllib as an external dependency, and we are working to make this change in our package. While compiliing btllib we are encountering this error:

meson.build:54:27: ERROR: Subproject exists but has no CMakeLists.txt file

A relevant GitHub Actions run can be found here.

Relates to https://github.com/Homebrew/homebrew-core/pull/130187

[parham-k]

Hi @p-linnane! Thanks for maintaining abyss on homebrew.

This is very likely due to the fact that the git submodules haven't been downloaded alongside btllib's code.

I see you're using https://github.com/bcgsc/btllib/archive/refs/tags/v1.6.0.tar.gz as btllib's url in the formula. I guess GitHub does not package submodules with tags' archives. Switching this url to https://github.com/bcgsc/btllib/releases/download/v1.6.0/btllib-1.6.0.tar.gz should resolve the issue.

[p-linnane]

Hello! Thank you for the pointer. This indeed got the build furthur, but it build is still failing. Here is the new error:

In file included from /private/tmp/abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/recipes/mi_bloom_filter.cpp:1:
../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/include/btllib/mi_bloom_filter.hpp: In member function 'void btllib::MIBloomFilter<T>::insert_id(const uint64_t*, const T&)':
../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/include/btllib/mi_bloom_filter.hpp:452:3: error: 'uint' was not declared in this scope; did you mean 'rint'?
  452 |   uint rand = std::rand(); // NOLINT
      |   ^~~~
      |   rint
../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/include/btllib/mi_bloom_filter.hpp:456:26: error: invalid operands of types 'int()' and 'const uint64_t' {aka 'const long long unsigned int'} to binary 'operator^'
  456 |     T random_num = (rand ^ hashes[i]) % count;
      |                     ~~~~ ^ ~~~~~~~~~
      |                     |              |
      |                     int()          const uint64_t {aka const long long unsigned int}
[50/59] Compiling C++ object libbtllib.a.p/_private_tmp_abyss-20230505-67356-y5ra6u_abyss-2.3.6_btllib_src_btllib_bloom_filter.cpp.o
[51/59] Compiling C++ object recipes/indexlr.p/_private_tmp_abyss-20230505-67356-y5ra6u_abyss-2.3.6_btllib_recipes_indexlr.cpp.o
[52/59] Compiling C++ object recipes/randseq.p/_private_tmp_abyss-20230505-67356-y5ra6u_abyss-2.3.6_btllib_recipes_randseq.cpp.o
[53/59] Compiling C++ object wrappers/python/_btllib.so.p/btllib_wrap.cxx.o
FAILED: wrappers/python/_btllib.so.p/btllib_wrap.cxx.o
g++-13 -Iwrappers/python/_btllib.so.p -Iwrappers/python -I../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/wrappers/python -I/opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/include/python3.11 -Iinclude -I../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/include -I. -I../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib -I../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/subprojects/cpptoml/include -Isubprojects/cpptoml/__CMake_build -I../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/subprojects/cpptoml/__CMake_build -Isubprojects/cpptoml -I../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/subprojects/cpptoml -fdiagnostics-color=always -Wall -Winvalid-pch -Wextra -Wpedantic -Werror -std=c++17 -O3 -fopenmp -isystem../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/subprojects/sdsl-lite -isystemsubprojects/sdsl-lite -isystem../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/subprojects/sdsl-lite/__CMake_build -isystemsubprojects/sdsl-lite/__CMake_build -isystem../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/subprojects/sdsl-lite/include -isystem../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/subprojects/sdsl-lite/__CMake_build/external/libdivsufsort/include -isystemsubprojects/sdsl-lite/__CMake_build/external/libdivsufsort/include -isystem../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/subprojects/sdsl-lite/__CMake_build/include -isystemsubprojects/sdsl-lite/__CMake_build/include -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk -w -MD -MQ wrappers/python/_btllib.so.p/btllib_wrap.cxx.o -MF wrappers/python/_btllib.so.p/btllib_wrap.cxx.o.d -o wrappers/python/_btllib.so.p/btllib_wrap.cxx.o -c ../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/wrappers/python/btllib_wrap.cxx
In file included from ../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/wrappers/python/btllib_wrap.cxx:4041:
../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/include/btllib/mi_bloom_filter.hpp: In member function 'void btllib::MIBloomFilter<T>::insert_id(const uint64_t*, const T&)':
../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/include/btllib/mi_bloom_filter.hpp:452:3: error: 'uint' was not declared in this scope; did you mean 'rint'?
  452 |   uint rand = std::rand(); // NOLINT
      |   ^~~~
      |   rint
../abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/include/btllib/mi_bloom_filter.hpp:456:26: error: invalid operands of types 'int()' and 'const uint64_t' {aka 'const long long unsigned int'} to binary 'operator^'
  456 |     T random_num = (rand ^ hashes[i]) % count;
      |                     ~~~~ ^ ~~~~~~~~~
      |                     |              |
      |                     int()          const uint64_t {aka const long long unsigned int}
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "/private/tmp/abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/./compile", line 182, in <module>
    sp.run(
  File "/opt/homebrew/Cellar/python@3.11/3.11.3/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['\n    if [ -f /tmp/btllib-build-cAbawUSfT6GorAdGG4TVQ/venv/bin/activate ]; then\n      . /tmp/btllib-build-cAbawUSfT6GorAdGG4TVQ/venv/bin/activate;\n    fi &&\n    \n    meson setup --buildtype release  -Db_ndebug=true --prefix=/private/tmp/abyss-20230505-67356-y5ra6u/abyss-2.3.6/btllib/install /tmp/btllib-build-cAbawUSfT6GorAdGG4TVQ &&\n    cd /tmp/btllib-build-cAbawUSfT6GorAdGG4TVQ &&\n    ninja install &&\n    cd .. &&\n    if [ -f /tmp/btllib-build-cAbawUSfT6GorAdGG4TVQ/venv/bin/activate ]; then\n      deactivate;\n    fi\n  ']' returned non-zero exit status 1.

[parham-k]

It seems that btllib is not fully compatible with the recently released GCC 13. I'll track the work required to fix this in a separate issue. In the meantime, is it possible to use clang or an older version of gcc with abyss?

In the abyss formula, it is specified that clang is not supported due to lack of openmp support. What if we add libomp to the list of dependencies like this line in btllib's formula?

An old version of btllib is packaged in brewsci/bio/btllib, I don't recommend using that as a permanent solution here, we're waiting for the latest releases to be merged into homebrew-bio.

[p-linnane]

I reworked the formula to use GCC 12 and encountered the same error unfortunately.

brewsci is not part of the official Homebrew organization, so I'm unable to merge that PR for you unfortunately.

[parham-k]

It's any version of GCC, then.

Temporary fix would be switching to clang. I'll release btllib 1.6.1 whenever the fix in #95 is merged.

[p-linnane]

I've confirmed patching 1.6.0 with that commit fixes the problem. Unfortunately, abyss is throwing an error, so that will need to be debugged separately. I've pushed it to our CI to run through all the OS and architecture combinations.

ld: in ../Assembly/libassembly.a(libcommon.a), archive member 'libcommon.a' with length 52248 is not mach-o or llvm bitcode file '../Assembly/libassembly.a' for architecture arm64
collect2: error: ld returned 1 exit status