memory usage & statistics optimization

kwai / blaze

Blazing-fast query execution engine speaks Apache Spark language and has Arrow-DataFusion at its core.

Apache License 2.0

968 stars 90 forks source link

Closed richox closed 1 month ago

richox commented 1 month ago

optimize sort-merge join memory usage

use more precise memory usage tracking for consumers

use record batch memory size as shuffle data size