[TASK][MEDIUM] Hive engine supports Apache Hive 4.0.0

[pan3793]

What's the level of this task?

MEDIUM

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

Search before creating

• [X] I have searched in the task list and found no similar tasks.

Mentor

• [ ] I have sufficient expertise on this task, and I volunteer to be a mentor of this task to guide contributors through the task.

Skill requirements

• Scala
• Hive/MR/Tez
• YARN
• Hadoop Security / Kerberos (optional)

Background and Goals

Currently, the Kyuubi Hive engine supports

• Cloudera Hive 2.1.1-cdh6
• Apache Hive 2.3
• Apache Hive 3.1

in both local and YARN deploy mode, and both SIMPLE and KERBEROS security mode.

Apache Hive released 4.0.0 on 29 March 2024 https://hive.apache.org/general/downloads/, we'd like to add support for this latest version too.

Implementation steps

change code/test/GHA workflows and make CI happy.

Additional context

Introduction of 2024H1 Kyuubi Code Contribution Program

This may help for testing https://github.com/awesome-kyuubi/hadoop-testing

[slfan1989]

@pan3793 Can I assign this issue? Thank you!

[pan3793]

@slfan1989 thanks, assigned. BTW, I don't know how difficult to support Hive 4, the task level will be adjusted based on the real efforts.

[slfan1989]

@slfan1989 thanks, assigned. BTW, I don't know how difficult to support Hive 4, the task level will be adjusted based on the real efforts.

Thank you for your explanation! I will try my best to complete it.

[Madhukar525722]

Hi @pan3793 and @slfan1989 are you guys working on this? Can we work together?