sanger-tol / blobtoolkit

Nextflow pipeline for BlobToolKit for Sanger ToL production suite
https://pipelines.tol.sanger.ac.uk/blobtoolkit
MIT License
10 stars 0 forks source link

Updates in place and broken Nextflow job cache #90

Closed muffato closed 2 months ago

muffato commented 5 months ago

Description of feature

In the version 0.2.0 of the pipeline, the job cache only functions up to the BLOBTOOLKIT_CREATEBLOBDIR process. Something downstream must be modifying some input file or parameter, breaking the cache mechanism.

My guess is that BLOBTOOLKIT_UPDATEBLOBDIR and others are updating the blobdir in place and therefore modifying its timestamp, which is an element considered by Nextflow when checking processess against the cache.

Even though the remaining processes don't take that long to rerun, we should make this cleaner.