justanhduc / task-spooler

A scheduler for GPU/CPU tasks
https://justanhduc.github.io/2021/02/03/Task-Spooler.html
GNU General Public License v2.0
273 stars 24 forks source link

GPU selection scheme #55

Open jasam-sheja opened 7 months ago

jasam-sheja commented 7 months ago

Is there a way to suggest beforehand which GPUs to allocate for a job?

My use case is I'm sharing a server with others, but unfortunately, they don't want to use task-spooler, and they hardcode their GPU IDs. We have set a soft division of the GPUs where it's okay to tap into their GPUs if I'm using all of mine. For example, a server with 8 GPUs and I got 0-3, and they have 4-7. I want to run five jobs with a GPU each, and they want to run 2. They'll always use their GPUs in the following order: 4,5,6,7. So, I want to set my preferred order as 0,1,2,3,7,6,5,4. By doing so, I don't get in their way.