search

pgiri / dispy

Distributed and Parallel Computing Framework with / for Python
https://dispy.org
Other
266 stars 55 forks source link

Launch one job on one node with many CPUs/cores #172

Closed max810 closed 5 years ago

max810 commented 5 years ago

Is there a way to launch ONLY ONE job simultaneously per node while using all the CPUs/cores on the node?

So, when on the node the first job is launched, it uses all the available CPUs/cores, it finishes, returns result and ONLY THEN the next job is started on that node.

pgiri commented 5 years ago

If your computation somehow can use all cores on a node, then you can by either starting dispynode with -n 1 (so all computations only use one processor on a node) or by giving parameter nodes to JobCluster with instances of NodeAllocate where cpus is given as 1, or instances of subclass of NodeAllocate overriding allocate method that returns 1, etc.

max810 commented 5 years ago

Thanks, with cpus=1 eveything seems to work as intended.