asafgal
commented
3 years ago This is a weird way to manage priorities.. Usually HPCs will limit the number of running jobs, or resources used, and not the number of queued jobs..
Anyhow, you can run a single step of the solve by using --step 1 (or 2 or 3). Note that you have to wait for one step to fully finish before running the next one.
Another way is to run by ‘graph’ number. A ‘graph’ in this context is a group of videos that are solved together. If you divided your experiment into few such graphs, you can use the --glist option, which accept an integer enumeration of these graphs. The default is to group by video subdirectories, but you can configure it as you like.
On 6 Apr 2021, at 14:56, Jana Mach @.***> wrote:
The HPC server I am using has certain limits per user (100 schedules jobs, 3 days max per job). The solve step in my case generated more than 100 jobs, causing some of the jobs getting cancelled. Since the solve step is a three step process, I figured I can start each step manually, e.g.:
janamach
Hi :-)
The HPC server I am using has certain limits per user (100 schedules jobs, 3 days max per job). The
solvestep in my case generated more than 100 jobs, causing some of the jobs getting cancelled. Since thesolvestep is a three step process, I figured I can start each step manually, e.g.:$ sbatch path/to/hpc_solve1.shIs this a reasonable way to do it? Is there a different way to go around the max job per user thing?