search

apache / incubator-gluten

Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.
https://gluten.apache.org/
Apache License 2.0
1.21k stars 434 forks source link

Support "Complete" Hash aggregations #1250

Open zhouyuan opened 1 year ago

zhouyuan commented 1 year ago

Is your feature request related to a problem or challenge? Please describe what you are trying to do. Usually hashagg will use "partial" plus "final" modes, the complete mode is used when direct hashagg result is expected, e.g., when pushing partial hashagg thru join

Describe the solution you'd like Following vanilla Spark, currently the hashagg operator supports partial/final mode only, should extend the code to support completemode as well

Describe alternatives you've considered N/A

Additional context N/A

zhli1142015 commented 8 months ago

Will velox also enable the MergeTwoPhasesHashBaseAggregate and support complete mode agg? Thanks.

jackylee-ch commented 5 months ago

@zhouyuan We meet some performance problem with this issue. Are you working on this issue now? If not, I'm glad to continue this work.

zhouyuan commented 5 months ago

@jackylee-ch please go ahead on the fix.