[server][dvc] Enable parallel transfers for each partition during blob transfer.
Major Changes:
Previously, blob transfers occurred sequentially for each partition. Even in parallel subscription, the NettyP2PBlobTransferManager would only return a future once the transfer was complete for error handling, resulting in a blocking process. This PR updates the process to be non-blocking.
Minor Changes:
(1) In the event of a connection issue, we will no longer attempt a retry; instead, we will proceed to the next host.
(2) For unknown or unexpected errors, instead of abandoning the blob transfer and executing Kafka ingestion, the process will now move on to the next host for the blob transfer.
(3) If files transfer exceeds the allowed time limit, the P2PFileTransferServerHandler will send a timeout response and will cease sending any files.
How was this PR tested?
unit test and integration test.
Does this PR introduce any user-facing changes?
[x] No. You can skip the rest of this section.
[ ] Yes. Make sure to explain your proposed changes and call out the behavior change.
[server][dvc] Enable parallel transfers for each partition during blob transfer.
Major Changes: Previously, blob transfers occurred sequentially for each partition. Even in parallel subscription, the NettyP2PBlobTransferManager would only return a future once the transfer was complete for error handling, resulting in a blocking process. This PR updates the process to be non-blocking.
Minor Changes: (1) In the event of a connection issue, we will no longer attempt a retry; instead, we will proceed to the next host. (2) For unknown or unexpected errors, instead of abandoning the blob transfer and executing Kafka ingestion, the process will now move on to the next host for the blob transfer. (3) If files transfer exceeds the allowed time limit, the P2PFileTransferServerHandler will send a timeout response and will cease sending any files.
How was this PR tested?
unit test and integration test.
Does this PR introduce any user-facing changes?