search

apache / celeborn

Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.
https://celeborn.apache.org/
Apache License 2.0
862 stars 351 forks source link

[CELEBORN-1574] Speed up unregister shuffle by batch processing #2701

Open zaynt4606 opened 3 weeks ago

zaynt4606 commented 3 weeks ago

What changes were proposed in this pull request?

In order to speed up the resource releasing，this PR Unregister shuffle in batch;

Does this PR introduce any user-facing change?

No

How was this patch tested?

UT & Local cluster testing