The number of jobs submitted by auto run is out of control, there are 2 ways of reducing the number of jobs needed to be submitted:
Reduce the number of jobs for each iteration
Make use of cluster specific options for submitting jobs
The number of jobs for each iteration can only be pushed so far, it may be possible to reduce the number from 7 to 5 easily and from 5 to 3 with some more difficulty. This will only reduce the number of jobs to be queued by a small amount and would still require many jobs to be queued.
An alternative is to make use of cluster specific options for submitting jobs. The current system where all jobs must be submitted upfront came about because csf doesn't allow for submitting jobs from compute nodes, this restriction doesn't exist on ffluxlab and we can make use of submitting jobs on compute nodes to have 1 script per instance running at a time. On csf this won't work but a 'drop-n-compute' system has been implemented specifically for this scenario and we should make use of it.
The number of jobs submitted by auto run is out of control, there are 2 ways of reducing the number of jobs needed to be submitted:
The number of jobs for each iteration can only be pushed so far, it may be possible to reduce the number from 7 to 5 easily and from 5 to 3 with some more difficulty. This will only reduce the number of jobs to be queued by a small amount and would still require many jobs to be queued.
An alternative is to make use of cluster specific options for submitting jobs. The current system where all jobs must be submitted upfront came about because csf doesn't allow for submitting jobs from compute nodes, this restriction doesn't exist on ffluxlab and we can make use of submitting jobs on compute nodes to have 1 script per instance running at a time. On csf this won't work but a 'drop-n-compute' system has been implemented specifically for this scenario and we should make use of it.