apache / orc

Apache ORC - the smallest, fastest columnar storage for Hadoop workloads
https://orc.apache.org/
Apache License 2.0
675 stars 480 forks source link

orc CXX fail to build if libgtest-dev is installed (debian-like systems) #1791

Closed douardda closed 6 months ago

douardda commented 7 months ago

Trying to compile the cxx tools/libs, I have had linking errors. Here is a minimal dockerfile to reproduce the issue:

FROM debian:bookworm

RUN export DEBIAN_FRONTEND=noninteractive && \
  apt-get update && apt-get upgrade -y && \
  apt-get install -y \
  build-essential \
  cmake \
  gcc \
  g++ \
  git \
  libsasl2-dev \
  libssl-dev \
  make 

WORKDIR /root/
RUN git clone https://github.com/apache/orc /root/orc

CMD mkdir build && \
    cd build && \
    cmake ../orc -DBUILD_JAVA=OFF && \
    make package && \
    echo "OK without libgtest" && \
    mkdir /root/build_err && \
    cd /root/build_err && \
    DEBIAN_FRONTEND=noninteractive apt install -y libgtest-dev  && \
    cmake ../orc -DBUILD_JAVA=OFF && \
    make package && \
    echo "OK WITH libgtest"

Then:

$ docker build -t test_orc .
$ docker run --rm -ti  test_orc
[...]
OK without libgtest
[...]
Setting up googletest (1.12.1-0.2) ...
Setting up libgtest-dev:amd64 (1.12.1-0.2) ...
[...]
[  8%] Performing configure step for 'snappy_ep'
-- snappy_ep configure command succeeded.  See also /root/build_err/snappy_ep-prefix/src/snappy_ep-stamp/snappy_ep-configure-*.log
[  8%] Performing build step for 'snappy_ep'
CMake Error at /root/build_err/snappy_ep-prefix/src/snappy_ep-stamp/snappy_ep-build-RELWITHDEBINFO.cmake:49 (message):
  Command failed: 2

   'make'

  See also

    /root/build_err/snappy_ep-prefix/src/snappy_ep-stamp/snappy_ep-build-*.log

make[2]: *** [CMakeFiles/snappy_ep.dir/build.make:86: snappy_ep-prefix/src/snappy_ep-stamp/snappy_ep-build] Error 1
make[1]: *** [CMakeFiles/Makefile2:259: CMakeFiles/snappy_ep.dir/all] Error 2
make: *** [Makefile:166: all] Error 2

The second cmake session fails with a link error while processing the snappy_ep build step. The log file contains lots of 'undefined reference' errors like:

snappy_unittest.cc:(.text+0x5e04): undefined reference to `testing::Message::Message()'
[...]
dongjoon-hyun commented 7 months ago

It sounds like snappy source code issue, doesn't it? Do you happen to know the corresponding issue report on snappy project?

douardda commented 7 months ago

I just cloned and built snappy alone and it works just fine (both on master and last tag). I also noted that it seems orc embeds snappy 1.1.7 while latest release from a year ago is 1.1.10. But updating snappy version in cmake_modules/ThirdpartyToolchain.cmake does not help. So something else in the way the embedded snappy is built is involved...

I also noted that the version of googltest embedded in snappy (submodule) is release-1.8.0-3137-gb796f7d4 (aka b796f7d44681514f58a683a3a71ff17c94edb0c1) which I don't really get how its related from latest tagged versions of gtest (orc pin version 1.12.1 in this 3rd party cmake file; which is exactly the version shipped by the debian package).

douardda commented 7 months ago

I just cloned and built snappy alone and it works just fine (both on master and last tag). I also noted that it seems orc embeds snappy 1.1.7 while latest release from a year ago is 1.1.10. But updating snappy version in cmake_modules/ThirdpartyToolchain.cmake does not help.

Correction: the error is not the same; it fails at configuration time in this case. digging...

douardda commented 7 months ago

I just cloned and built snappy alone and it works just fine (both on master and last tag). I also noted that it seems orc embeds snappy 1.1.7 while latest release from a year ago is 1.1.10. But updating snappy version in cmake_modules/ThirdpartyToolchain.cmake does not help.

Correction: the error is not the same; it fails at configuration time in this case. digging...

Ah the issue is tgz assets available on snappy's gh release page for 1.1.10 and 1.1.9 do not work...

dongjoon-hyun commented 6 months ago

This is resolved via ORC-1618 .