Closed aryarm closed 6 years ago
perhaps we could also scale the number of threads used depending on the rule? not everything should use num_threads
threads, right?
this may be something that snakemake does already. I won't know until I look into it
it seems that scaling jobs is a little harder than I thought it would be
one way to do it would be to put an upper cap on memory usage and then increase the memory usage of a job after each attempt
but this doesn't seem like a great solution yet. let's table this issue
you can make the pipeline more robust to cluster mishaps using the
--restart-times
option in combination with a function that automatically scales the number of jobs depending on how often jobs have had to be restarted see the documentation for an explanation of how the scaling can be done