search

filodb / FiloDB

Distributed Prometheus time series database
Apache License 2.0
1.43k stars 225 forks source link

(misc-iceberg-export): Remove sortWithinPartitions to avoid spill(memory) and spill(disk) #1791

Closed nikitag55 closed 4 months ago

nikitag55 commented 4 months ago

Current behavior :

In case of large volume of data to be exported, with sortWithinPartitions() is causing spill(disk) & spill(memory).

New behavior :

Removing this operation will not do sorting within partitions and will not cause memory & disk spill.