Open nnop opened 6 years ago
slurm and mpi
some hpc sites:
interactive shell
nodes
-N, --nodes=<minnodes[-maxnodes]>
要注意,如果只用-N=1
申请的是最少nodes数目。
tasks
What does the --ntasks or -n tasks does in SLURM?
The
--ntasks
options specifies how many instances of your command are executed. In contrast the option--cpus-per-task
specify how many CPUs each task can use.
The -n, -c, and -N options control how CPUs and nodes will be allocated to the job. When specifying only the number of processes to run with -n, a default of one CPU per process is allocated. By specifying the number of CPUs required per task (-c), more than one CPU may be allocated per process. If the number of nodes is specified with -N, srun will attempt to allocate at least the number of nodes specified.
Combinations of the above three options may be used to change how processes are distributed across nodes and cpus. For instance, by specifying both the number of processes and number of nodes on which to run, the number of processes per node is implied. However, if the number of CPUs per process is more important then number of processes (-n) and the number of CPUs per process (-c) should be specified. srun will refuse to allocate more than one process per CPU unless --overcommit (-O) is also specified.
srun will attempt to meet the above specifications "at a minimum." That is, if 16 nodes are requested for 32 processes, and some nodes do not have 2 CPUs, the allocation of nodes will be increased in order to meet the demand for CPUs. In other words, a minimum of 16 nodes are being requested. However, if 16 nodes are requested for 15 processes, srun will consider this an error, as 15 processes cannot run across 16 nodes.
The term "task" in this context can be thought of as a "process". Therefore, a multi-process program (e.g. MPI) is comprised of multiple tasks. And a multi-threaded program is comprised of a single task, which can in turn use multiple CPUs. In Slurm, tasks are requested with the
--ntasks
flag. CPUs, for the multithreaded programs, are requested with the--cpus-per-task
flag.Hybrid (MPI+OpenMP) programs For the most predictable performance for hybrid codes, you will need to use all three of the
--ntasks
,--cpus-per-task
, and--nodes
flags, where--ntasks
equals the number of MPI tasks,--cpus-per-task
equals the number ofOMP_NUM_THREADS
and--nodes
is the number of nodes required to fit--ntasks
*--cpus-per-task
.
short answer is yes. Long answer is that a task is a Slurm allocation unit and a process is a Linux running process ; processes are created by the commands present in the submission script and mapped the to CPUs associated with the tasks allocated to the job. –- damienfrancois
multithreaded tasks