search

apache / celeborn

Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.
https://celeborn.apache.org/
Apache License 2.0
862 stars 351 forks source link

[CELEBORN-1579] Fix the memory leak of result partition #2706

Closed reswqa closed 2 weeks ago

reswqa commented 2 weeks ago

What changes were proposed in this pull request?

Don't register RemoteShuffleResultPartition to partition manager because we don't store resource in TM.

In theory, we could also fix this on the Flink side by no longer registering partitions to the partition manager for remote/cluster partitions, but that will only be released after Flink 2.0 at the soon.

Why are the changes needed?

RemoteShuffleResultPartition will be registered to the partition manager(at setup phase). Since it's a cluster partition(resources are not stored on the Flink TM), Flink does not trigger the resource releasing over TM.

In a session cluster, the partition object is leaked. As a more serious consequence, the failure of the partition to release will result in the idle TM not being reclaimed by the Flink resource manager.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Self tested in a session cluster.

mridulm commented 2 weeks ago

+CC @venkata91

SteNicholas commented 2 weeks ago

@reswqa, thanks for contribution. Merged to main(v0.6.0) and branch-0.5(v0.5.2).