WSL (Ubuntu 22.04) build fails due to HDFS not supporting clang++-16

[KVeschgini]

What's wrong Clean build on wsl (Ubuntu 22.04.1 LTS) fails. First there is an issue with pthread. I resolved it by adding

set(CMAKE_THREAD_LIBS_INIT "-lpthread")
set(CMAKE_HAVE_THREADS_LIBRARY 1)
set(CMAKE_USE_WIN32_THREADS_INIT 0)
set(CMAKE_USE_PTHREADS_INIT 1)
set(THREADS_PREFER_PTHREAD_FLAG ON)

to the top level CMakeLists.txt

Then the build failed due to HDFS not supporting clang++-16.

How to reproduce

git clone --recurse-submodules git@github.com:timeplus-io/proton.git
# -- Edit CMakeLists.txt as described if necessary
cd proton
mkdir build
cd build
cmake ..

Error message

-- Enable kerberos for HDFS
CMake Error at contrib/libhdfs3-cmake/CMake/Platform.cmake:40 (MESSAGE):
  Unsupported compiler: "/usr/bin/clang++-16"
Call Stack (most recent call first):
  contrib/libhdfs3-cmake/CMakeLists.txt:26 (include)

-- Configuring incomplete, errors occurred!

[chenziliang]

Before we fix this, you can try

mkdir release_build
cd release_build
bash ../build.sh
ninja proton

[yokofly]

building Proton is taking quite a bit of time, sorry about this.

I'm planning to set up my WSL to give it a try. the HDFS can be disable directly -DENABLE_HDFS=OFF or use the build.sh mentioned above.

we shall not encounter the issue with Pthread, anyway, I'm going to give the build a shot today on WSL 2204.

[yokofly]

WSL is slower than I expected, and I encounter OOM (Out of Memory) errors many times. I eventually succeeded in building the project using ../build.sh with the commit ad3dc2b.

I used clang-17, but it didn't make much difference.

The only strange issue I faced was the threads library missing error. I encountered this as well, but it was resolved after cleaning the folder and trying again.

[KVeschgini]

I tried https://github.com/timeplus-io/proton/pull/797, I ran into the following error:

In file included from /mnt/c/Users/k.veschgini/dev/src/proton/contrib/muduo/muduo/net/EventLoop.cc:9:
In file included from /mnt/c/Users/k.veschgini/dev/src/proton/contrib/muduo/muduo/net/EventLoop.h:14:
In file included from /mnt/c/Users/k.veschgini/dev/src/proton/contrib/llvm-project/libcxx/include/atomic:526:
In file included from /mnt/c/Users/k.veschgini/dev/src/proton/contrib/llvm-project/libcxx/include/__thread/timed_backoff_policy.h:17:
In file included from /mnt/c/Users/k.veschgini/dev/src/proton/contrib/llvm-project/libcxx/include/__threading_support:35:
In file included from /mnt/c/Users/k.veschgini/dev/src/proton/cmake/linux/../../contrib/sysroot/linux-x86_64/x86_64-linux-gnu/libc/usr/include/pthread.h:22:
/mnt/c/Users/k.veschgini/dev/src/proton/contrib/muduo/muduo/net/endian.h:51:10: error: use of undeclared identifier 'htobe64'
  return htobe64(host64);
         ^
/mnt/c/Users/k.veschgini/dev/src/proton/contrib/muduo/muduo/net/endian.h:56:10: error: use of undeclared identifier 'htobe32'
  return htobe32(host32);
         ^
/mnt/c/Users/k.veschgini/dev/src/proton/contrib/muduo/muduo/net/endian.h:61:10: error: use of undeclared identifier 'htobe16'
  return htobe16(host16);
         ^
/mnt/c/Users/k.veschgini/dev/src/proton/contrib/muduo/muduo/net/endian.h:66:10: error: use of undeclared identifier 'be64toh'
  return be64toh(net64);
         ^
/mnt/c/Users/k.veschgini/dev/src/proton/contrib/muduo/muduo/net/endian.h:71:10: error: use of undeclared identifier 'be32toh'
  return be32toh(net32);
         ^
/mnt/c/Users/k.veschgini/dev/src/proton/contrib/muduo/muduo/net/endian.h:76:10: error: use of undeclared identifier 'be16toh'
  return be16toh(net16);
         ^
6 errors generated.
ninja: build stopped: subcommand failed.

[KVeschgini]

The issue with muduo arises because the /mnt/c/ directory is accessed in a case-insensitive way, leading the #include directive to incorrectly include the Endian.h file from muduo instead. I resolved this by turning on case sensitivity in that specific folder.

[jovezhong]

Thanks. Not sure whether you can send out a PR to improve this

[yokofly]

splendid :) thanks