sanger-bentley-group / gps-pipeline

Nextflow Pipeline for processing Streptococcus pneumoniae sequencing raw reads (FASTQ files) by the GPS Project (Global Pneumococcal Sequencing Project)
GNU General Public License v3.0
12 stars 4 forks source link

suggestions to singularity preflight #100

Closed Phuong-Le closed 4 months ago

Phuong-Le commented 4 months ago

https://github.com/sanger-bentley-group/gps-pipeline/blob/fcbcece2c278fa56792850a68f1325f6c6246089/main.nf#L32

@HarryHung thanks very much for sharing your code. A good idea to predownload images prior to running workflows.

Here, it seems that you're searching the first config file for a 'container' flag. However, if someone else were to adopt this and run with multiple config files, this would be quite unstable.

Nextflow seems to have the workflow.container variable that would be useful in this case?

Better still, I think the preflight code deserves its own git repo, so you can use it in multiple pipelines with something like a git submodule. Would this be something you're interested in? Let's discuss this further?

HarryHung commented 4 months ago

@Phuong-Le Thanks for the suggestion, your presumption is correct. That line was an ugly workaround to address workflow.container not working when more than one container/image is being used in the workflow back then: https://github.com/nextflow-io/nextflow/issues/1758

It seems the issue has been fixed now:

First Nextflow stable release includes this fix is v23.10.0

I will:

Regarding making a repo for it, I can consider that. Side note, I personally prefer using git subtree to ensure git clone still work as expected for typical users.

Phuong-Le commented 4 months ago

ah okay I didn't know workflow.container wasn't working back in the day, and yeah can confirm workflow.container is working as I tested it before submitting the issue.

Definitely keep me posted - thanks for suggesting git subtree

HarryHung commented 4 months ago

@Phuong-Le

Fixed in dev, will be included in next release.

If I extract the singularityPreflight function into another repository, I will create another ticket for it.