Closed ksenia-krasheninnikova closed 4 months ago
nf-core lint
overall result: Passed :white_check_mark: :warning:Posted for pipeline commit 59be6c8
+| ✅ 130 tests passed |+
#| ❔ 19 tests were ignored |#
!| ❗ 3 tests had warnings |!
If you just want to use the latest version of hifiasm, I can see one here: https://quay.io/repository/biocontainers/hifiasm?tab=tags&tag=latest
If you want to build your own version of hifiasm, you don't need to use wave.
Like I said yesterday in Slack, wave can build docker image on the fly using a Dockefile or just a conda package on the fly, you don't need to store them somewhere.
Thanks @gq1!
The reason why we need a combined container is described in the PR description (see above) - we need samtools and hifiasm in the same package.
The on-the-fly functionality hasn't been implemented yet: https://github.com/nf-core/modules/pull/4940
We can wait until it's implemented or we can host the image built with wave-cli and use it in the 'container' section.
Hi both, I think the current recommendation is to use mulled containers: they're exactly there for combining multiple Conda packages, and they've been used many times.
Like you said, @ksenia-krasheninnikova , integrating wave into nf-core is still work-in-progress on their side (that may resume soon, cf https://github.com/nf-core/modules/pull/4080#issuecomment-2075518454 )
If we want to manually build containers via wave and store them somewhere without waiting for the nf-core implementation, we'd need a bit of tooling to simplify that. We also need to be clear on when doing that vs when building a custom container in our docker-images repo. For instance, it feels like the cramfilter_bwamem2_minimap2_samtools_perl
container could be managed by Wave as well. @gq1 : think about that, the pros and cons, etc, and we can discuss when I'm back
Both nf-core PRs only try to pre-build the containers using Conda or Spack packages for modules? https://github.com/nf-core/modules/pull/4940 https://github.com/nf-core/modules/pull/4080
I didn't try, maybe it just works if we have a local or modified nf-core modules when they only have conda environment or Spack file with wave?
According to this discussion on slack it should be possible to add a “conda” directive and -with-wave and it should just work
Not sure how you created the wave image currently, manually using conda and wave?
The CI job failed when converting Docker image to SIF? You can try to run with Docker profile first?
FATAL: While making image from oci registry: error fetching image to cache: while building SIF from layers: conveyor failed to get: no descriptor found for reference
It may be because the Wave container registry only keeps images for 1 week and it's now been deleted.
I've just created the wave image this week. The pipeline works fine for me locally with singularity profile. With docker docker/24.0.2-c1 hifiasm module fails for me locally on one of the farm nodes with this error:
Command error:
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
touch: cannot touch '.command.trace': Read-only file system
@gq1 The image was created with this:
wave-1.3.0-linux-x86_64 --conda-package hifiasm=0.19.8 --conda-package samtools=1.20
You have a similar yaml file like this? https://github.com/nf-core/modules/blob/25f2b00a02ae84e44644e4ff94ec56130f984f79/modules/nf-core/bowtie/align/environment.yml Then only conda directive here: https://github.com/nf-core/modules/blob/25f2b00a02ae84e44644e4ff94ec56130f984f79/modules/nf-core/bowtie/align/main.nf#L5
Can you update the description of this PR?
For this purpose wave-cli was used and a new wave container was generated.
Since all containers kept by the native wave infrastructure are deleted in a week - should it be moved from
wave.seqera.io/wt/73ac3caec075/wave/build:hifiasm-0.19.8_samtools-1.20--1f6824530f0d0ad5
to the private repo and in this case do we have a ToL repo for that?
Otherwise, there is a way of building a container with wave inside of the project with Dockerfile: https://www.nextflow.io/docs/latest/wave.html#build-module-containers
The hifiasm patch facilitates piping in HiC data directly from CRAM using
samtools fastq
without creating intermediate FASTQ files. This update is aimed at updating hifiasm to the latest version (0.19.8). For this purpose wave-cli was used and a new wave container was generated.Since all containers kept by the native wave infrastructure are deleted in a week - should it be moved from
wave.seqera.io/wt/73ac3caec075/wave/build:hifiasm-0.19.8_samtools-1.20--1f6824530f0d0ad5
to the private repo and in this case do we have a ToL repo for that?
Otherwise, there is a way of building a container with wave inside of the project with Dockerfile: https://www.nextflow.io/docs/latest/wave.html#build-module-containers
UPD: In the course of working on the PR it was revealed that nf-core wave functionality has not been fully implemented to be built into the workflow. The combined image of hifiasm and samtools was created with wave-cli and will be hosted for now at quay.io/sanger-tol. The command used to build the image:
wave-1.3.0-linux-x86_64 --conda-package hifiasm=0.19.8 --conda-package samtools=1.20
PR checklist
nf-core lint
).nextflow run . -profile test,docker --outdir <OUTDIR>
).docs/usage.md
is updated.docs/output.md
is updated.CHANGELOG.md
is updated.README.md
is updated (including new tool citations and authors/contributors).