apache / kyuubi

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.
https://kyuubi.apache.org/
Apache License 2.0
2.11k stars 915 forks source link

[CI] Specify JDK 11 for spark engine on kubernetes IT #6763

Closed wForget closed 1 month ago

wForget commented 1 month ago

:mag: Description

Issue References ๐Ÿ”—

Spark Engine On Kubernetes Integration Test always fails after #6739

The default java version of Ubuntu 24.04 is OpenJDK 21: https://discourse.ubuntu.com/t/ubuntu-24-04-lts-noble-numbat-release-notes/39890#openjdk , this is a version higher than that supported by spark 3.5.X.

Describe Your Solution ๐Ÿ”ง

Specify JDK 11 for spark engine on kubernetes it.

Types of changes :bookmark:

Test Plan ๐Ÿงช

test ga: https://github.com/wForget/kyuubi/actions/runs/11441231714/job/31828818049


Checklist ๐Ÿ“

Be nice. Be informative.

pan3793 commented 1 month ago

I feel like that maven.compiler.release should be set to the minimal supported JDK version, for our case, it should be 8 instead of following the compile JDK version.

wForget commented 1 month ago

I feel like that maven.compiler.release should be set to the minimal supported JDK version, for our case, it should be 8 instead of following the compile JDK version.

Should this version be consistent with the jdk version used to build the spark image (apache/spark:3.5.2)?

codecov-commenter commented 1 month ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 0.00%. Comparing base (1e9d68b) to head (aee3933). Report is 3 commits behind head on master.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #6763 +/- ## ====================================== Coverage 0.00% 0.00% ====================================== Files 684 684 Lines 42353 42354 +1 Branches 5775 5776 +1 ====================================== - Misses 42353 42354 +1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

bowenliang123 commented 1 month ago

BTW, the default JDK on Ubuntu 24.04 is version 21. https://launchpad.net/ubuntu/noble/amd64/default-jdk

bowenliang123 commented 1 month ago

I feel like that maven.compiler.release should be set to the minimal supported JDK version, for our case, it should be 8 instead of following the compile JDK version.

SGTM. Testing it in #6766.

wForget commented 1 month ago

I feel like that maven.compiler.release should be set to the minimal supported JDK version, for our case, it should be 8 instead of following the compile JDK version.

SGTM. Testing it in #6766.

Thank you, that's a better way, I will close this

bowenliang123 commented 1 month ago

As #6766 requires further polishment and investigation, we still need this patch as a workaround to pass the CI tests.

bowenliang123 commented 1 month ago

Thanks, merged to master (1.10.0)