search

Azure / hpcpack

The repo to track public issues for Microsoft HPC Pack product.
MIT License
29 stars 11 forks source link

Question: How to make sure a task is allocated only on a single node #4

Open syagev opened 4 years ago

syagev commented 4 years ago

I'd like each task in my job to be run on a single node, however I was only able to find in the documentation the -SingleNode argument that applies to entire jobs. Is there a way to achieve this for tasks?

Also, what is the meaning of allocation of, e.g. cores, across multiple nodes for the same task? If a task is allocated 10 cores from node 1 and 20 cores from node 2 - will the exact same command line simply run twice, once on each node?

YutongSun commented 4 years ago

Currently single node task is not supported. It is a valid requirement.

mkbosmans commented 10 months ago

This would indeed be very useful functionality. Our use case is a job with several instances of a multithreaded application. This is shared memory parallelism, so all cores for a single task should be allocated on the same node. Each task with the job should be allowed to be on different nodes though.

Seems like a very basic requirement, but somehow MS HPC seems either hardly maintained at all or just focused on MPI-like workloads.

daanscheltens commented 9 months ago

Indeed a very nice addition that we are also looking for.

I would in this case also prefer to option to contsrain my task to a single socket as that is more memory friendly. I know I can assing a whole socket to a task, but than I have many Idle cores. On my 24 core socket I would like to be able to run 4 tasks of 6 cores each. These tasks are allowed to run on any socket of any node, however all 6 cores should be on the same socket.

Setting tasks to stick to a single node is definately step one and a minimum requirement for these tasks to run. However, being able to also constrain them to a socket would help in memory efficiency.

YutongSun commented 9 months ago

HPC Pack 2019 Update 2 (6.2.7756) supports single node tasks. For single socket tasks, we will check the implementation in the next release.