For simulations in which there are many replicates, but for which each replicate takes a small amount of time to run (<10 seconds), parallelizing on a cluster via job arrays with one replicate per task ID may involve a lot of overhead. A solution would be the ability to run multiple replicates on a single job array task.
Avi-Kenny
commented
2 years ago
For simulations in which there are many replicates, but for which each replicate takes a small amount of time to run (<10 seconds), parallelizing on a cluster via job arrays with one replicate per task ID may involve a lot of overhead. A solution would be the ability to run multiple replicates on a single job array task.