[CELEBORN-1721][CIP-12] Support HARD_SPLIT in PushMergedData - Githubissues

apache / celeborn

Apache Celeborn is an elastic and high-performance service for shuffle and spilled data.

https://celeborn.apache.org/

Apache License 2.0

896 stars 361 forks source link

[CELEBORN-1721][CIP-12] Support HARD_SPLIT in PushMergedData #2924

Open jiang13021 opened 1 week ago

jiang13021 commented 1 week ago

What changes were proposed in this pull request?

As title.

Why are the changes needed?

https://docs.google.com/document/d/1Jaix22vME0m1Q-JtTHF9WYsrsxBWBwzwmifcPxNQZHk/edit?tab=t.0#heading=h.iadpu3t4rywi (Thanks to @cfmcgrady @littlexyw @ErikFang @waitinfuture @RexXiong @FMX for their efforts on the proposal)

Does this PR introduce any user-facing change?

The response of pushMergedData has been modified, however, the changes are backward compatible.

How was this patch tested?

UT: org.apache.celeborn.service.deploy.cluster.PushMergedDataHardSplitSuite

jiang13021 commented 6 days ago

Thank you, @RexXiong and @FMX , for your review comments. I’ve made the necessary changes. Could you please take another look?