Open architkulkarni opened 1 year ago
Hi, I'm a bot from the Ray team :)
To help human contributors to focus on more relevant issues, I will automatically add the stale label to issues that have had no activity for more than 4 months.
If there is no further activity in the 14 days, the issue will be closed!
You can always ask for help on our discussion forum or Ray's public slack channel.
Summary
This RFC proposes a change in Ray's terminology to distinguish between "worker nodes" (hardware resources) and "worker processes" (individual processes executing tasks/actors), in an effort to reduce recurring user confusion and improve the overall clarity of Ray's API and documentation.
Proposal
The proposed changes are as follows:
Rename all instances of
worker
referring to worker nodes toworker_node
in the Ray API, VM launcher, KubeRay YAMLs, and other relevant locations. This includes fields such asmax_workers
andmin_workers
in the VM launcher YAML, which would be renamed tomax_worker_nodes
andmin_worker_nodes
.Maintain backward compatibility by supporting both
worker
andworker_node
in our interfaces.Update the documentation to always use "worker node" when referring to hardware resources, and clarify the distinction between "worker node" and "worker".
Alternatives Considered
One alternative would be to introduce a different term, such as "executor", to replace "worker" in the context of worker processes. However, there seems to be a consensus that "worker" is a term commonly used and understood in the machine learning community to refer to individual execution units (processes), as seen in libraries like PyTorch and TensorFlow. Therefore, maintaining the term
worker
for worker processes and introducingworker_node
for hardware resources is the preferred approach.Status
Comments, feedback, and alternative suggestions are welcomed for this proposal. After 2 weeks, we'll make this change in a backward-compatible manner as part of the Ray 2.6 release.
cc @pcmoritz @richardliaw @GokuMohandas @rkooo567 @gvspraveen @scottsun94