search

BCCDC-PHL / fluviewer-nf

Nextflow pipeline for influenza A whole genome sequence analysis
0 stars 1 forks source link

Add support for singularity/apptainer containers #35

Open dfornika opened 4 months ago

dfornika commented 4 months ago

Build singularity/apptainer images for this pipeline.

See our BCCDC-PHL/routine-assembly pipeline for an example of how to build apptainer containers and publish to the GitHub container registry.

Also investigate using the Sequera Wave system for building containers, if it can be easily integrated.

jpalmer37 commented 3 months ago

I created a draft branch to explore the use of Apptainer for managing pipeline dependencies / execution environments.

Summary of Changes

Created three .def files that define Apptainer images for our three pipeline environments. Right now, this setup builds, stores, and runs images locally, not yet on a cloud repository. All testing was performed on Nextflow version 24.04, which supports apptainer containerization.

Problem

While the initial trimming and filtering steps worked smoothly, the depth_normalization step of the pipeline is extremely slow (might even be broken) when using apptainer. Process was hanging for multiple hours. Might need to investigate tuning the process or apptainer images to get this step working.