It4innovations / hyperqueue

Scheduler for sub-node tasks for HPC systems with batch scheduling
https://it4innovations.github.io/hyperqueue
MIT License
275 stars 21 forks source link

Add a command for deploying workers #714

Open Kobzol opened 3 months ago

Kobzol commented 3 months ago

Sometimes HyperQueue is not in charge of creating Slurm/PBS allocations, but it is already started in an environment with pre-allocated nodes, and the users then just need to deploy a HQ worker on each node. It would be nice if HQ could help with this, and support the most obvious use-case, deploying through SSH.

The command could look like this:

$ hq worker deploy-ssh <nodefile> <worker-start-params>

It would read the node file, and essentially run hq worker start <worker-start-params> on each node.