Xtra-Computing / FedTree

A tree-based federated learning system (MLSys 2023)
https://fedtree.readthedocs.io/en/latest/index.html
Apache License 2.0
140 stars 38 forks source link

Feature Request: add support for "wait-for-ready" #69

Closed Junyi-99 closed 9 months ago

Junyi-99 commented 11 months ago

Thank you for your FedTree contribution!

During the recent distributed deployment, we observed that the client does not wait for the server to be ready before initiating communication. Instead, it throws a gRPC failure and go into the progress of "training".

According to gRPC document:

When an RPC is created when the channel has failed to connect to the server, without Wait-for-Ready it will immediately return a failure;

Perhaps it would be beneficial to consider adding the "Wait-for-Ready" feature to the project, as it would enhance its robustness.

Once again, we appreciate your involvement in the project.