Closed JohnHellgren closed 6 years ago
Hi John, we had some issues with building on Docker Hub and I had to set up Docker Cloud to get this to run. I will have a look and push a 1.0 image in a couple of minutes.
Started pushing the image, this will take a while to finish.
You should be able to get the image now - let me know if otherwise, I will close this now!
Yes, apologies for this - we have the docker build setup automated so I expected it to magically appear in the night after the release. We only noticed this morning that it hadn't triggered due to a confusing mix of docker web infrastructures. All fixed (and hopefully future-proofed) now! Let us know if you run into any problems.
Thanks
I still have the bind point issue
Command error:
WARNING: Non existent bind point (directory) in container: '/c3se'
WARNING: Non existent bind point (directory) in container: '/apps'
WARNING: Non existent bind point (directory) in container: '/usr/share/lmod/lmod'
WARNING: Non existent bind point (directory) in container: '/var/hasplm'
WARNING: Non existent bind point (directory) in container: '/local'
WARNING: Could not chdir to home: /c3se/users/johnhe/Hebbe
Do I have to add it manually this time as well? Last time I solved it by asking the support at our computer cluster Do you have a link to the singularity container? Like with the last pipeline: https://hub.docker.com/r/scilifelab/ngi-rnaseq/builds/
Hi John,
Yes, if you can't enable overlays then you will need to manually add those bind points for every new singularity image that you want to use.
x-ref: https://github.com/SciLifeLab/NGI-RNAseq/issues/174#issuecomment-386239084 and https://github.com/SciLifeLab/NGI-RNAseq/issues/174#issuecomment-386571706
The nfcore/rnaseq v1.0 tagged docker container is visible here: https://cloud.docker.com/swarm/nfcore/repository/docker/nfcore/rnaseq/tags
So your Singularity
bootstrap file should look something like this:
Bootstrap: docker
From: nfcore/rnaseq:1.0
%post
mkdir ${SINGULARITY_ROOTFS}/c3se
mkdir ${SINGULARITY_ROOTFS}/local
mkdir ${SINGULARITY_ROOTFS}/apps
mkdir ${SINGULARITY_ROOTFS}/usr/share/lmod/lmod
mkdir ${SINGULARITY_ROOTFS}/var/hasplm
mkdir ${SINGULARITY_ROOTFS}/var/opt/thinlinc
$ singularity build ngi-rnaseq-hebbe.simg Singularity
Note that we're aware that this process is a pain and we have been actively trying to develop a helper tool to make this process easier, see https://github.com/nf-core/tools/issues/69 - hopefully the next time there's a release we'll have a helper tool to automate this process for you.
Phil
Hi @JohnHellgren,
@remiolsen also suggested trying to use the -B
option with Singularity. However, we're not sure if this works or not on systems were overlayFS is not enabled. We don't have access to any such systems, so this is tough for us to test!
If you're interested, it would be great if you could give it a try...
1) Create a nextflow config file, eg. singularity-c3se.config.txt
:
singularity {
runOptions = '-B /c3se -B /local -B /apps -B /usr/share/lmod/lmod -B /var/hasplm -B /var/opt/thinlinc'
}
2) Run the pipeline with the original Singularity container, but specifying the above config:
nextflow run nf-core/rnaseq -c singularity-c3se.config.txt ....
* Modified launch script: [rna_seq_script.txt](https://github.com/nf-core/rnaseq/files/2313892/rna_seq_script.txt) (requires you to create `/c3se/NOBACKUP/users/johnhe/rnaseq/nf-core/config/singularity-c3se.config.txt` from above).
If this does work, then there's no need for the creation of a custom Singularity container, which would be much better. We can also add all of this to the hebbe
profile in the pipeline, so you won't need to specify it in the future (and it will work for all hebbe users).
Let me know what you think!
Cheers,
Phil
Hello,
I noticed that the 1.0 release is out, but I cannot download the singularity container for this version
I tired this command:
But the I got this
running the pipeline without pulling the container first doesnt work for c3se
I ran the pipeline like this:
sbatch rna_seq_script.sh
rna_seq_script.shBest, John