search

aryarm / as_analysis

A complete Snakemake pipeline for detecting allele specific expression in RNA-seq
MIT License
10 stars 9 forks source link

re-attempt failed jobs and scale resource usage #43

Closed aryarm closed 6 years ago

aryarm commented 6 years ago

you can make the pipeline more robust to cluster mishaps using the --restart-times option in combination with a function that automatically scales the number of jobs depending on how often jobs have had to be restarted see the documentation for an explanation of how the scaling can be done

aryarm commented 6 years ago

perhaps we could also scale the number of threads used depending on the rule? not everything should use num_threads threads, right? this may be something that snakemake does already. I won't know until I look into it

aryarm commented 6 years ago

it seems that scaling jobs is a little harder than I thought it would be

one way to do it would be to put an upper cap on memory usage and then increase the memory usage of a job after each attempt

but this doesn't seem like a great solution yet. let's table this issue