search

apache / arrow-nanoarrow

Helpers for Arrow C Data & Arrow C Stream interfaces
https://arrow.apache.org/nanoarrow
Apache License 2.0
173 stars 38 forks source link

Tests fail linking against libarrow.so built in Conda environment #254

Open pitrou opened 1 year ago

pitrou commented 1 year ago

I have a Conda environment where I do Arrow development. When I try to build nanoarrow in that same environment I get linking errors in tests.

[23/25] Linking CXX executable utils_test
FAILED: utils_test utils_test[1]_tests.cmake /home/antoine/arrow/nanoarrow/build-debug/utils_test[1]_tests.cmake 
: && /home/antoine/mambaforge/envs/pyarrow/bin/g++ -g  CMakeFiles/utils_test.dir/src/nanoarrow/utils_test.cc.o -o utils_test  -Wl,-rpath,/home/antoine/mambaforge/envs/pyarrow/lib  libnanoarrow.a  lib/libgtest_maind.a  /home/antoine/mambaforge/envs/pyarrow/lib/libarrow.so.1300.0.0  lib/libgtestd.a  -lpthread && cd /home/antoine/arrow/nanoarrow/build-debug && /home/antoine/mambaforge/envs/pyarrow/bin/cmake -D TEST_TARGET=utils_test -D TEST_EXECUTABLE=/home/antoine/arrow/nanoarrow/build-debug/utils_test -D TEST_EXECUTOR= -D TEST_WORKING_DIR=/home/antoine/arrow/nanoarrow/build-debug -D TEST_EXTRA_ARGS= -D TEST_PROPERTIES= -D TEST_PREFIX= -D TEST_SUFFIX= -D TEST_FILTER= -D NO_PRETTY_TYPES=FALSE -D NO_PRETTY_VALUES=FALSE -D TEST_LIST=utils_test_TESTS -D CTEST_FILE=/home/antoine/arrow/nanoarrow/build-debug/utils_test[1]_tests.cmake -D TEST_DISCOVERY_TIMEOUT=5 -D TEST_XML_OUTPUT_DIR= -P /home/antoine/mambaforge/envs/pyarrow/share/cmake-3.26/Modules/GoogleTestAddTests.cmake
/home/antoine/mambaforge/envs/pyarrow/bin/../lib/gcc/x86_64-conda-linux-gnu/13.1.0/../../../../x86_64-conda-linux-gnu/bin/ld: warning: libcrypto.so.3, needed by /home/antoine/mambaforge/envs/pyarrow/lib/libarrow.so.1300.0.0, not found (try using -rpath or -rpath-link)
[...]
/home/antoine/mambaforge/envs/pyarrow/bin/../lib/gcc/x86_64-conda-linux-gnu/13.1.0/../../../../x86_64-conda-linux-gnu/bin/ld: /home/antoine/mambaforge/envs/pyarrow/lib/libarrow.so.1300.0.0: undefined reference to `BrotliEncoderCompressStream'
[etc.]

nanoarrow is configured using:

cmake .. -DCMAKE_BUILD_RPATH=${CONDA_PREFIX}/lib -DCMAKE_BUILD_TYPE=debug -DNANOARROW_BUILD_TESTS=on -GNinja

Note that all libraries do resolve from the command line:

$ lddtree /home/antoine/mambaforge/envs/pyarrow/lib/libarrow.so.1300.0.0
libarrow.so.1300.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libarrow.so.1300.0.0 (interpreter => none)
    libcrypto.so.3 => /home/antoine/mambaforge/envs/pyarrow/lib/libcrypto.so.3
    libssl.so.3 => /home/antoine/mambaforge/envs/pyarrow/lib/libssl.so.3
    libbrotlienc.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libbrotlienc.so.1
    libbrotlidec.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libbrotlidec.so.1
    libbrotlicommon.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libbrotlicommon.so.1
    liborc.so => /home/antoine/mambaforge/envs/pyarrow/lib/liborc.so
    libprotobuf.so.32 => /home/antoine/mambaforge/envs/pyarrow/lib/libprotobuf.so.32
    libutf8proc.so.2 => /home/antoine/mambaforge/envs/pyarrow/lib/libutf8proc.so.2
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1
    libgoogle_cloud_cpp_storage.so.2 => /home/antoine/mambaforge/envs/pyarrow/lib/libgoogle_cloud_cpp_storage.so.2
    libbz2.so.1.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libbz2.so.1.0
    liblz4.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/liblz4.so.1
    libsnappy.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libsnappy.so.1
    libz.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libz.so.1
    libzstd.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libzstd.so.1
    libaws-cpp-sdk-config.so => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-cpp-sdk-config.so
    libaws-cpp-sdk-transfer.so => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-cpp-sdk-transfer.so
    libaws-cpp-sdk-identity-management.so => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-cpp-sdk-identity-management.so
    libaws-cpp-sdk-cognito-identity.so => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-cpp-sdk-cognito-identity.so
    libaws-cpp-sdk-sts.so => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-cpp-sdk-sts.so
    libaws-cpp-sdk-s3.so => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-cpp-sdk-s3.so
    libaws-cpp-sdk-core.so => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-cpp-sdk-core.so
    libre2.so.10 => /home/antoine/mambaforge/envs/pyarrow/lib/libre2.so.10
    libabsl_str_format_internal.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_str_format_internal.so.2206.0.0
    libgoogle_cloud_cpp_rest_internal.so.2 => /home/antoine/mambaforge/envs/pyarrow/lib/libgoogle_cloud_cpp_rest_internal.so.2
    libgoogle_cloud_cpp_common.so.2 => /home/antoine/mambaforge/envs/pyarrow/lib/libgoogle_cloud_cpp_common.so.2
    libabsl_time.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_time.so.2206.0.0
    libabsl_strings.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_strings.so.2206.0.0
    libabsl_strings_internal.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_strings_internal.so.2206.0.0
    libabsl_int128.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_int128.so.2206.0.0
    libabsl_civil_time.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_civil_time.so.2206.0.0
    libabsl_time_zone.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_time_zone.so.2206.0.0
    libabsl_bad_variant_access.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_bad_variant_access.so.2206.0.0
    libabsl_base.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_base.so.2206.0.0
    libabsl_spinlock_wait.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_spinlock_wait.so.2206.0.0
    libabsl_bad_optional_access.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_bad_optional_access.so.2206.0.0
    libabsl_throw_delegate.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_throw_delegate.so.2206.0.0
    libabsl_raw_logging_internal.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_raw_logging_internal.so.2206.0.0
    libabsl_log_severity.so.2206.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libabsl_log_severity.so.2206.0.0
    libcrc32c.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libcrc32c.so.1
    libcurl.so.4 => /home/antoine/mambaforge/envs/pyarrow/lib/libcurl.so.4
        libnghttp2.so.14 => /home/antoine/mambaforge/envs/pyarrow/lib/./libnghttp2.so.14
        libssh2.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/./libssh2.so.1
        libgssapi_krb5.so.2 => /home/antoine/mambaforge/envs/pyarrow/lib/./libgssapi_krb5.so.2
            libkrb5.so.3 => /home/antoine/mambaforge/envs/pyarrow/lib/././libkrb5.so.3
            libk5crypto.so.3 => /home/antoine/mambaforge/envs/pyarrow/lib/././libk5crypto.so.3
            libcom_err.so.3 => /home/antoine/mambaforge/envs/pyarrow/lib/././libcom_err.so.3
            libkrb5support.so.0 => /home/antoine/mambaforge/envs/pyarrow/lib/././libkrb5support.so.0
            libkeyutils.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/././libkeyutils.so.1
            libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2
    libaws-crt-cpp.so => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-crt-cpp.so
    libaws-c-mqtt.so.1.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-mqtt.so.1.0.0
    libaws-c-event-stream.so.1.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-event-stream.so.1.0.0
    libaws-c-s3.so.0unstable => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-s3.so.0unstable
    libaws-c-auth.so.1.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-auth.so.1.0.0
    libaws-c-http.so.1.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-http.so.1.0.0
    libaws-c-io.so.1.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-io.so.1.0.0
    libs2n.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libs2n.so.1
    libaws-c-compression.so.1.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-compression.so.1.0.0
    libaws-c-cal.so.1.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-cal.so.1.0.0
    libaws-c-sdkutils.so.1.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-sdkutils.so.1.0.0
    libaws-checksums.so.1.0.0 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-checksums.so.1.0.0
    libaws-c-common.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libaws-c-common.so.1
    libstdc++.so.6 => /home/antoine/mambaforge/envs/pyarrow/lib/libstdc++.so.6
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
    libgcc_s.so.1 => /home/antoine/mambaforge/envs/pyarrow/lib/libgcc_s.so.1
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
    ld-linux-x86-64.so.2 => /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
pitrou commented 1 year ago

@kou Do you have any idea how to solve this?

pitrou commented 1 year ago

Note that setting LD_LIBRARY_PATH works around the issue.

paleolimbot commented 1 year ago

Thank you for reporting!

I can reproduce this on Ubuntu 22.04 with:

conda create --name nanoarrow-verify-rc
conda activate nanoarrow-verify-rc
conda config --set channel_priority strict

mamba install -c conda-forge compilers cmake make git xsimd
curl https://dlcdn.apache.org/arrow/arrow-12.0.1/apache-arrow-12.0.1.tar.gz | \
  tar -zxf -
mkdir arrow-build && cd arrow-build
cmake ../apache-arrow-12.0.1/cpp -DARROW_FILESYSTEM=ON -DARROW_WITH_ZLIB=ON -DARROW_DEPENDENCY_SOURCE=CONDA
cmake --build . --parallel=$(nproc)
cmake --install . --prefix=${CONDA_PREFIX}
cd ..

git clone https://github.com/apache/arrow-nanoarrow.git arrow-nanoarrow
cd arrow-nanoarrow
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Debug -DNANOARROW_BUILD_TESTS=ON -DCMAKE_BUILD_RPATH=${CONDA_PREFIX}/lib
cmake --build .
kou commented 1 year ago

(I'm creating an environment to reproduce this...)

kou commented 1 year ago

I couldn't reproduce this on Ubuntu 22.04 based on https://app.vagrantup.com/bento/boxes/ubuntu-22.04 ... I used https://repo.anaconda.com/miniconda/Miniconda3-py311_23.5.2-0-Linux-x86_64.sh instead of mamba. I used command lines provided by https://github.com/apache/arrow-nanoarrow/issues/254#issuecomment-1632847590 but I replaced mamba with conda.

paleolimbot commented 1 year ago

I spent some more time on this and found that if you restart the terminal, activate the requisite conda environment, and do a fresh configure + build of nanoarrow, the problem goes away (however, if you don't do a fresh configure + build of nanoarrow, the linking error persists). It appears that you can't build + install Arrow C++ from the same terminal session (or perhaps conda activation) where you configure and build nanoarrow that links to Arrow C++.

As a workaround in case this doesn't fix the problem, #267 lets you pass -DNANOARROW_ARROW_STATIC=ON and statically link Arrow into the tests. We did this anyway on Windows...the PR just generalized the option for times like this.

pitrou commented 1 year ago

It appears that you can't build + install Arrow C++ from the same terminal session (or perhaps conda activation) where you configure and build nanoarrow that links to Arrow C++.

I build Arrow C++ and nanoarrow from different terminal instances, so I doubt this is the issue?

vyasr commented 7 months ago

I haven't been able to reproduce this. I've tried it out with both current and a couple of older versions of arrow, but using the latest nanoarrow. Would it make sense to close this for now but reopen if and when the issue reoccurs? I'm happy to take a look at it when that happens since I'm doing a bit of work right now to improve nanoarrow's CMake anyway.