Closed ezherman closed 2 years ago
Hi Ezra,
thank you for reporting this bug.
It seems some of the issue you're getting (write permissions, potentially unstable network connectivity leading to failed downloads) could be linked to the container implementation.
Unfortunately, I have never used pantagruel
from a Singularity container, or at least not this version. The Dockerfile has been designed for building a Docker image and only tested as a Docker container.
I understand that Docker may not be supported on your computer cluster for security issues; in fact, the Docker image installation option has been made mostly for usage on a cloud.
That said, our IT team at the Sanger has made a Singularity image for pantagruel
, which (I think) was working correctly last time I checked. I could try and see what they have done to the Dockerfile or what options were used when building the image to make it work as a Singularity container.
Could you please share the commands you used to build the image and to run the container?
With regards to the files being not within reach of write permissions, there is probably something (either a folder or some file) where permissions belong to the root user from the container (even though everything shows as belonging to you). if that is so, once the container exits, i'm not sure you can even re-access these files through another container, as the root users would technically be different users... that's me making hypotheses here, as I don't have deep knoledge of container system, expecially Singularity ones. I can only suggest two options:
rm
commands through the singularity container, to re-create the permission environment that crated the filessudo
privileges.I hope this helps you a bit.
Best wishes,
Florent
Hi Florent,
Thank you for offering to check with the IT team how the Singularity container was implemented. That would be really helpful.
I am running Singularity 3.8.5 in a conda environment on Ubuntu 20.04 through Windows Subsystem for Linux. When using the Slurm HPC (University of York's Viking cluster), I am using Singularity 3.5.3, installed by the IT team.
I am unable to reproduce the failure to delete pseudomonas_test_db
. Now when the fetching fails, the folder can be deleted with rm -rf
. I am sorry that I can't provide you with more details on this issue!
I am however able to reproduce the failure to fetch assemblies. After running the pantagruel -i (...) all
command a few times, it seems like the failure is not associated with a particular assembly. In fact, I even had a couple of fetches that completed without any issues. Could it be that a temporary network failure causes the fetch command to fail?
I have now encountered a new issue: if fetching does complete, pantagruel freezes at task 2. Is this something you've encountered before? See the output below.
[2022-02-21 14:28:08] Pantagruel pipeline task 2: align homologous protein sequences and reverse-translate aligned proteins into aligned coding sequences.
Create new task folder '/mnt/c/Users/elh605/pantagruel_test_loc/pantagruel_pipeline/pseudomonas_test_db/02.gene_alignments'
[2022-02-21 14:28:13]-- step 1: alignment of nr protein families with clustalo and GNU parallel
Academic tradition requires you to cite works you base your article on.
When using programs that use GNU Parallel to process data for publication
please cite:
O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
;login: The USENIX Magazine, February 2011:42-47.
This helps funding further development; AND IT WON'T COST YOU A CENT.
If you pay 10000 EUR you should feel free to use GNU Parallel without citing.
To silence this citation notice: run 'parallel --citation'.
Hereby the commands used to run the container and pantagruel:
mkdir pantagruel_pipeline
cd pantagruel_pipeline
git clone --recurse-submodules https://github.com/flass/pantagruel.git
singularity pull docker://flass/pantagruel-dep:master-latest
singularity run -B $PWD:$PWD pantagruel-dep_master-latest.sif
pantagruel/install_interproscan.sh $PWD/ $PWD/
# run with pseudomonas assemblies, which sometimes fetches without any issues
# however pantagruel freezes in step 1 of task 2 (alignment of nr protein families with clustalo and GNU parallel)
pantagruel -d pseudomonas_test_db -r ./ -f PANTAGFAM -I elh605 -L refseq_list_test init
pantagruel -i ./pseudomonas_test_db/environ_pantagruel_pseudomonas_test_db.sh all
Hi Ezra,
OK - it's good news that the file permission issue is not occurring any more.
The file fetching problem is likely something to do with instable network, which is in turn likely to do with the interface between the singularity container and the lftp
software. It could be that some parameterising of the singularity run or image building can solve it... I've tested again the singularity installation on our cluster at Sanger and this steps completes fine, so there might be a trick to make it work stably (it could also be something at the level of how the whole cluster is parameterised and how it allows singularity containers to access the internet... maybe you can ask your IT admin team).
Regarding the block in task 2, that's a completely different bug. Not really a bug in fact, just the need to run once the command parallel --citation
on the system to acknowledge the authors before you can use the parallel
command freely. in a container environment it is a problem because every container is like a new system so it has to be acknowledged every time a new container is created/run. It's something that came up in GNU parallel after I last built the Docker image, but your Singularity image has it.
I've introduce automated acknowledgement in commits 1c47117 and 501e894 (master) and 8d8c10f (usingGeneRax).
So if you re-build your Singularity image now with last code it should work fine - or if not please let me know, but ideally in a separate issue.
I consider this issue resolved, however I'll try to document on how to have stable network using Singularity, and post again here if I find.
Thank you Florent!
Hi Florent,
I was hoping to get your advice on issues I'm having when downloading assemblies using the
-L
flag. There are two issues:I initiated within a Singularity container using the following command:
pantagruel -d pseudomonas_test_db -r ./ -f PANTAGFAM -I elh605 -L refseq_test_list init
The file
refseq_test_list
contains:I then run the full workflow:
pantagruel -i pseudomonas_test_db/environ_pantagruel_pseudomonas_test_db.sh -N 4 all
The download then fails with
GCF_002208645.1
:Following this, I am unable to delete the
pseudomonas_test_db
folder usingrm -rf pseudomonas_test_db/
:Giving myself write and execute permissions over the files and the
pseudomonas_test_db
directory did not solve the issue. I am able to delete the files on my personal computer withsudo
, however I cannot do this on my university's HPC cluster.I was wondering whether:
pseudomonas_test_db
folder.Thanks in advance!