search

vinisalazar / metaphor

Metaphor: a general-purpose workflow for assembly and binning of metagenomes
https://metaphor-workflow.readthedocs.io/
Other
37 stars 3 forks source link

Restarting pipeline from broken step #34

Closed francesco-ricci closed 1 year ago

francesco-ricci commented 1 year ago

Hello there,

I had a problem with disk space while running the pipeline, my tmp folder got full. Since metaphor was running for a few days I was wondering if there is a way to recover the analysis from where it stopped.

Thanks Francesco

francesco-ricci commented 1 year ago

Actually, the problem is not in my TMPDIR but in another folder where snakemake writes tmp files.

Specifically: OSError: [Errno 28] No space left on device: '/home/ubuntu/.cache/snakemake/snakemake/source-cache/runtime-cache/tmpnazv5090'

vinisalazar commented 1 year ago

Hello @francesco-ricci, sorry that I didn't reply to you here earlier.

Metaphor restarts from where it left off by default. This is a feature of Snakemake, which keeps track of the output files that have been (correctly) generated. You use the exact same command that you use to start your analysis, to resume it.

If, when trying to resume your analysis, you get an error like this:

Building DAG of jobs...
Error: Directory cannot be locked. Please make sure that no other Snakemake process is trying to create the same files in the following directory:
/path/to/workflow/
If you are sure that no other instances of snakemake are running on this directory, the remaining lock was likely caused by a kill signal or a power loss. It can be removed with the --unlock argument.

You should run Metaphor with the --unlock, using the -e or --extras flag, which add extra flags to the Snakemake command:

`metaphor execute -e " --unlock"``

This will unlock your analysis directory and, if you use the same command that you used to start your analysis, it should work.

I will close this issue for now, but please feel free to reopen it if you continue to have problems.

Best, V