apache / incubator-uniffle

Uniffle is a high performance, general purpose Remote Shuffle Service.
https://uniffle.apache.org/
Apache License 2.0
369 stars 143 forks source link

[Improvement] `rss.client.assignment.shuffle.nodes.max` should not be less than `rss.data.replica` #1196

Open xianjingfeng opened 11 months ago

xianjingfeng commented 11 months ago

Code of Conduct

Search before asking

What would you like to be improved?

For current assignment strategy, if rss.client.assignment.shuffle.nodes.max less than rss.data.replica, the duplicate shuffle server will be included in the replicas.

How should we improve?

Check if rss.client.assignment.shuffle.nodes.max less than rss.data.replica in the client side. We can modify the following method.

https://github.com/apache/incubator-uniffle/blob/10e8e3d2eb553146d9729e5dac2bde20f71c3358/common/src/main/java/org/apache/uniffle/common/util/RssUtils.java#L310C22-L319

Are you willing to submit PR?

somelovelanguage commented 7 months ago

Kindly to ask if there is somebody working for this issue? If not, i'm glad to undertake it.