search

apache / paimon

Apache Paimon is a lake format that enables building a Realtime Lakehouse Architecture with Flink and Spark for both streaming and batch operations.
https://paimon.apache.org/
Apache License 2.0
2.27k stars 911 forks source link

[Feature] Reduce memory use when scan manifests #907

Closed wxplovecc closed 1 year ago

wxplovecc commented 1 year ago

Search before asking

Motivation

Too many memory use of List objects when scan manifests 8071c29043ea96545213d58082a4ac98

Solution

Change List to Iterator

Anything else?

No response

Are you willing to submit a PR?

JingsongLi commented 1 year ago

Fixed by https://github.com/apache/incubator-paimon/pull/1072