search

Bishop-Laboratory / RLPipes

RLPipes: A standardized R-loop-mapping pipeline.
https://anaconda.org/bioconda/rlpipes
MIT License
3 stars 1 forks source link

locked directory when trying to resume pipeline #89

Closed markdunning closed 2 years ago

markdunning commented 2 years ago

Hi,

Our HPC has a hard limit of 4 days for any job, and my rlpipes run exceeded this and was killed.

As I understand it, I should be able to re-run the pipeline and it should pick-up from where it got to? However, on the pipeline re-run I get the following:-

Error: Directory cannot be locked. Please make sure that no other Snakemake process is trying to create the same files in the following directory:
/mnt/fastdata/md1mjdx/INDUCESEQ_drip/rlpipes_out
If you are sure that no other instances of snakemake are running on this directory, the remaining lock was likely caused by a kill signal or a power loss. It can be removed with the --unlock argument.

I'm afraid I'm completely new to snakemake. How can I remove the locked directory?

Many thanks,

Mark

millerh1 commented 2 years ago

Hi @markdunning -- thank you for reporting this issue! The problem is related to how snakemake handles jobs which did not finish successfully. To unlock the directory and continue the run, you will need to supply the following additional arguments to RLPipes:

RLPipes run <other args> -s "{'unlock': True, 'force_incomplete': True}"

As explained in the RLPipes usage, the -s flag allows you to pass arguments as a dict to the snakemake python API. In this case, you pass {'unlock': True} to unlock the locked snakemake directory. I also think you may need 'force_incomplete': True to ensure any files produced by incomplete jobs are re-created.

Please let me know if this does not solve the issue you describe.

markdunning commented 2 years ago

Apologies for the slow response. Thanks so much for the information. I haven't had chance to try it out yet, but will let you know how it goes.

markdunning commented 2 years ago

This did indeed work. However, I had to run it once to unlock the directory and once again to actually run the pipeline.

millerh1 commented 2 years ago

Good to know! Thank you for following up about it! I believe that is the intended behavior of snakemake