ray-project / rayfed

A multiple parties joint, distributed execution engine based on Ray, to help build your own federated learning frameworks in minutes.
https://rayfed.readthedocs.io
Apache License 2.0
91 stars 20 forks source link

[Multi-Job] The KV Actor increase management burdens of Multi-Job #135

Closed NKcqx closed 8 months ago

NKcqx commented 1 year ago

Client mode's internal_kv is implemented by creating a named Actor for transporting kv data between Ray cluster and client driver.

This implementation:

  1. forces the driver to manage its lifecycle, e.g. kill it when fed.shutdown;
  2. In multi-job mode, each Job has to maintain its own KV actor with a unique actor name.

However, none of the above are unnecessary because Ray already provides an official interface for KV accessing under client mode, no Actor is needed.

Replacing KV Actor to the official interface