search

Urban-M4 / snakemake-wrf

Random notes, scripts, et cetera
Apache License 2.0
2 stars 1 forks source link

Be able to submit more than one experiment without waiting for the other to finish #14

Open ClaireDons opened 4 months ago

ClaireDons commented 4 months ago

At the moment the workflow waits for each experiment to finish before submitting another. Would be nice to be able to submit one after another without waiting.

ClaireDons commented 4 months ago

Possibly related, but snakemake exits with an error if the job times out.

Peter9192 commented 4 months ago

You could do that by removing the wait option from https://github.com/Urban-M4/misc/blob/eab22fde05dea647f0be0b1d993c2bb65c81baa0/workflows/snakemake/Snakefile#L174

But then you cannot do any other task after it that depends on it. Currently there's a 'cleanup workdir' task after WRF. If you don't wait, this will start right after the job submission. But perhaps we can make it a two-step workflow, that first runs up to WRF job submission, and you need to invoke a separate rule to do cleanup.