Submit to specific worker or to auto-alloc

[unkcpz]

Is that possible to have multiple auto-alloc and during submit, tell which target to use. My use case is I want to alloc nodes from two slurm partition, one from regular partition that can run longer jobs but usually need to wait and one from the partition that for test purpose. Maybe this is a feature that already exist?

[Kobzol]

Hi, in HQ, you should be tying tasks to workers using abstract resources, rather than explicitly specifying which worker should execute which task. If you want to specify that some tasks should only run on workers that live for a longer time, you can use a time request, for example hq submit --time-request 2h ..., this will ensure that the task will only be executed on a worker that has at least 2 hour runtime available.

That being said, if you do want to explicitly select a specific kind of a worker, you can do that using resources. For example, you can require the resource foo (an arbitrary name) for a task, with hq submit --resource foo=1, and then only provide this resource on specific workers, using hq worker start --resource 'foo=9999' or hq alloc add --resource 'foo=9999' ....

[unkcpz]

Thanks for explanation @Kobzol! Makes a lot sense, and by adding another worker without less queue time the jobs are running on there immediately as expected.