[VL] Umbrella issue for sort-based shuffle - Githubissues

apache / incubator-gluten

Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.

https://gluten.apache.org/

Apache License 2.0

1.14k stars 415 forks source link

[VL] Umbrella issue for sort-based shuffle #6896

Open marin-ma opened 4 weeks ago

marin-ma commented 4 weeks ago

Description

Initial patch: https://github.com/apache/incubator-gluten/pull/6475 (TPCDS SF3T 40000 partitions 1.13x vs hash)
- follow up: https://github.com/apache/incubator-gluten/pull/6579
- follow up: https://github.com/apache/incubator-gluten/pull/6628
[Performance] https://github.com/apache/incubator-gluten/pull/6639 (1.25x)
[Performance] https://github.com/apache/incubator-gluten/pull/6727 (1.3x)
[Performance] https://github.com/apache/incubator-gluten/pull/6782 (1.33x) (Pending on Velox PR)
[UI] https://github.com/apache/incubator-gluten/pull/6799
[Bug fix #6822] https://github.com/apache/incubator-gluten/pull/6832
[Bug fix #6863] https://github.com/apache/incubator-gluten/pull/6869

FelixYBW commented 3 weeks ago

Good summary! thank you