search

flink-extended / flink-remote-shuffle

Remote Shuffle Service for Flink
Apache License 2.0
191 stars 57 forks source link

[FRS-27] Support to write shuffle data to local shuffle worker first #30

Closed wsry closed 2 years ago

wsry commented 2 years ago

What is the purpose of the change

This PR solves #27 . It Introduces a new PartitionPlacementStrategy which always tries to select the local worker first. For scenarios where network bandwidth is the bottleneck, this can save network bandwidth.

Brief change log

Add a new PartitionPlacementStrategy which always tries to select the local worker first.

Verifying this change

This change added tests.

wsry commented 2 years ago

Merged.