apache / incubator-gluten

Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.
https://gluten.apache.org/
Apache License 2.0
1.21k stars 436 forks source link

[Doc][Build]Ubuntu 24.04 support - Test + Doc #7977

Open yabinma opened 3 hours ago

yabinma commented 3 hours ago

Background:

In the Getting-Started document, it's mentioned that Ubuntu 20.04/22.04 are supported. While Ubuntu 24.04 LTS was released on April 2024 and it's a long term supported version.

This ticket is for tracking the test on Ubuntu 24.04 and it will list the issues met and it will enhance the document once the test done.

Build Test:

prerequisite: cmake, openjdk-8, ninja, mvn

sudo apt update & sudo apt install -y build-essential openjdk-8-jdk cmake ninja-build

gcc v13.2 will be installed and this is the major difference with Ubuntu 22.04(gcc v11.4)

maven download link : https://maven.apache.org/download.cgi for example maven v3.8.8 : https://dlcdn.apache.org/maven/maven-3/3.8.8/binaries/apache-maven-3.8.8-bin.tar.gz

After download, tar xvzf apache-maven-3.8.8-bin.tar.gz unzip to a folder for example, /opt/apache-maven-3.8.8

After install completion, edit /etc/profile to add JAVA_HOME and M2_HOME. vi /etc/profile, to add,

export M2_HOME=/opt/apache-maven-3.8.8
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export PATH=$M2_HOME/bin:$JAVA_HOME/bin:$PATH

source /etc/profile to load the profile for current user.

Build: As the getting-started doc mentioned, ./dev/buildbundle-veloxbe.sh is used to build in one script. Two ENV variables are recommended, for example,

TREAT_WARNINGS_AS_ERRORS=0 NUM_THREADS=4 ./dev/buildbundle-veloxbe.sh

By default, all warnings are treated as errors. TREAT_WARNINGS_AS_ERRORS=0 is to disable it and to avoid build error because of warnings. NUM_THREADS=X is to specify the CPU core in compile. This is to avoid the OOM issue in compile.

build issue 1:

/opt/gitspace/incubator-gluten/ep/build-velox/build/velox_ep/velox/functions/prestosql/InPredicate.cpp: In lambda function:
/opt/gitspace/incubator-gluten/ep/build-velox/build/velox_ep/velox/functions/prestosql/InPredicate.cpp:94:50: error: call of overloaded ‘contains(<brace-enclosed initializer list>)’ is ambiguous
   94 |         const bool found = uniqueValues_.contains({arg.get(), row});

It does not happen with gcc v11.4. With gcc 13.2, the implicit type caused the overload error. A code fix is opened in PR: https://github.com/facebookincubator/velox/pull/11573 The fix is to specify the type explicitly.

build issue 2: <To be added>

yabinma commented 3 hours ago

Hello Team, anyone please help assign the ticket to me? I will keep adding the test result and any fix required. Thanks.