apache / incubator-gluten

Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.
https://gluten.apache.org/
Apache License 2.0
1.22k stars 437 forks source link

[GLUTEN-7126][DNM][VL][1.2] Port #6698 #7525 #7560 for Uniffle bug fix #7994

Open weiting-chen opened 2 days ago

weiting-chen commented 2 days ago

What changes were proposed in this pull request?

Port #6698 #7525 #7560 to fix Uniffle related bug fix in branch-1.2

How was this patch tested?

local

github-actions[bot] commented 2 days ago

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

PHILO-HE commented 2 days ago

Seem CI failure is related. @SteNicholas, @wForget

 /root/.m2/repository/org/apache/uniffle/rss-client-spark3-shaded/0.8.0/rss-client-spark3-shaded-0.8.0.jar(org/apache/spark/shuffle/RssShuffleManager.class): warning: Cannot find annotation method 'value()' in type 'SuppressFBWarnings': class file for edu.umd.cs.findbugs.annotations.SuppressFBWarnings not found
Error:  /__w/incubator-gluten/incubator-gluten/gluten-uniffle/velox/src/main/java/org/apache/spark/shuffle/writer/VeloxUniffleColumnarShuffleWriter.java:[199,16] error: cannot find symbol
Error:    symbol:   class RssException
Error:    location: class VeloxUniffleColumnarShuffleWriter<K,V>
PHILO-HE commented 1 day ago

@weiting-chen, the uniffle test fails which should be related.

github-actions[bot] commented 19 hours ago

https://github.com/apache/incubator-gluten/issues/7126

summaryzb commented 1 hour ago

@weiting-chen the changes of gluten-uniffle/velox/src/main/java/org/apache/spark/shuffle/writer/VeloxUniffleColumnarShuffleWriter.java is made in https://github.com/apache/incubator-gluten/pull/6579 should not be include in this pr. cc @PHILO-HE @jackylee-ch