[VL] Umbrella issue for sort-based shuffle - Githubissues

apache / incubator-gluten

Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.

https://gluten.apache.org/

Apache License 2.0

1.22k stars 439 forks source link

[VL] Umbrella issue for sort-based shuffle #6896

Open marin-ma opened 3 months ago

marin-ma commented 3 months ago

Description

Initial patch: https://github.com/apache/incubator-gluten/pull/6475 (TPCDS SF3T 40000 partitions 1.13x vs hash)
- follow up: https://github.com/apache/incubator-gluten/pull/6579
- follow up: https://github.com/apache/incubator-gluten/pull/6628
[Performance] https://github.com/apache/incubator-gluten/pull/6639 (1.25x)
[Performance] https://github.com/apache/incubator-gluten/pull/6727 (1.3x)
[Performance] https://github.com/apache/incubator-gluten/pull/6782 (1.33x) (Pending on Velox PR)
[UI] https://github.com/apache/incubator-gluten/pull/6799
[Bug fix #6822] https://github.com/apache/incubator-gluten/pull/6832
[Bug fix #6863] https://github.com/apache/incubator-gluten/pull/6869
[Performance] https://github.com/apache/incubator-gluten/pull/7897

FelixYBW commented 3 months ago

Good summary! thank you