Granular Worker management. - Githubissues

SCALE-MS / scale-ms

SCALE-MS design and development

GNU Lesser General Public License v2.1

4 stars 4 forks source link

Granular Worker management. #335

Open eirrgang opened 1 year ago

eirrgang commented 1 year ago

Follow-up to #318

[X] Move dispatch out of WorkflowManager
[X] Merge Queuer and manage_execution -> RPExecutor + manager_raptor
[X] Place "stop" command as part of ordinary shutdown.
[ ] Replace "dispatch" call stack with "executor" (migrate or remove old code, examples, tests)

Follow-up (need consultation)

[ ] Launch Worker according to work load.
[ ] Add a CPI command to start/stop Worker.
[ ] Separate the Worker lifetime from master lifetime.

Deferred

Solidify Executor role with a single implementation to manage the dispatching context (re-implement scalems.execution module)
refactor RuntimeManager / Executor collaboration
- implement Executor in terms of CPI/RPC spec to be specialized by RuntimeManagers.
- Executor and RuntimeManager collaborate to fulfil Futures.