search

ray-project / rayfed

A multiple parties joint, distributed execution engine based on Ray, to help build your own federated learning frameworks in minutes.
https://rayfed.readthedocs.io
Apache License 2.0
91 stars 20 forks source link

What's the meaning of the "cluster_config" ? #156

Open NKcqx opened 1 year ago

NKcqx commented 1 year ago 
          Now that we separate `ray.init` from `fed.init`,  there's no way to reach the cluster-level information, since each `fed.init` starts and only starts a job session.

Unless there's a global actor (or service job) that can break the job isolation and filter each job's tasks' invalid param type.

_Originally posted by @NKcqx in https://github.com/ray-project/rayfed/pull/140#discussion_r1263583091_

NKcqx commented 1 year ago 
          If we don't have the cluster config, why we not just use `config`? The question that we should answer before it getting finalized is whether we need the cluster config in the future at high level.

_Originally posted by @jovany-wang in https://github.com/ray-project/rayfed/pull/140#discussion_r1263675724_

NKcqx commented 1 year ago

Firstly, the fact is that user can't configure the Ray cluster in a rayfed job, since the initialization of Ray cluster has separated from RayFed, i.e. fed.init. I think the original semantic of "cluster_config" is "configure the cluster used in this job", in which case, it's a job-level config but containing all the non-business configurations.

jovany-wang commented 1 year ago

I think the original semantic of "cluster_config" is "configure the cluster used in this job", in which case, it's a job-level config but containing all the non-business configurations.

It sounds reasonable. Let's use the config key word as the job level configurations parameter name.

jovany-wang commented 11 months ago

What's the status of this PR?