Problems running (Dockerhub) Virsorter on a Server

[DanieleDeco]

Hi Simroux,

I have some problems running Dockerhub virsorter on a server, I get no output and these errors.

bioresource@bioresorce1 virsorter-run]$ docker run -v /home/bioresource/media/bay1/daniele/Virome/Virsorter/virsorter-data:/data -v /home/bioresource/media/bay1/daniele/Virome/Virsorter/virsorter-run:/wdir --rm discoenv/virsorter:v1.0.3 --db 2 --fna /wdir/contigs.fasta Dataset : VIRSorter Fna file : /wdir/contigs.fasta Db : 2 Wdir : /root Custom phages:

%#%#%#%#%# Processing VIRSorter....

Step 0.5 : /usr/local/bin/Scripts/Step_1_contigs_cleaning_and_gene_prediction.pl VIRSorter /root/fasta /root/fasta/input_sequences.fna 2 >> /root/log_out 2>> /root/log_err Step 0.8 : /usr/local/bin/hmmsearch --tblout Contigs_prots_vs_PFAMa.tab --cpu 8 -o Contigs_prots_vs_PFAMa.out --noali /data/PFAM_27/Pfam-A.hmm /root/fasta/VIRSorter_prots.fasta >> /root/log_out 2>> /root/log_err Step 0.9 : /usr/local/bin/hmmsearch --tblout Contigs_prots_vs_PFAMb.tab --cpu 8 -o Contigs_prots_vs_PFAMb.out --noali /data/PFAM_27/Pfam-B.hmm /root/fasta/VIRSorter_prots.fasta >> /root/log_out 2>> /root/log_err

### Revision 0

Out : Step 1.2 : /usr/local/bin/hmmsearch --tblout r_0/Contigs_prots_vs_New_clusters.tab --cpu 8 -o r_0/Contigs_prots_vs_New_clusters.out --noali r_0/db/Pool_clusters.hmm /root/fasta/VIRSorter_prots.fasta >> /root/log_out 2>> /root/log_err Step 1.3 : /usr/local/bin/blastall -p blastp -i /root/fasta/VIRSorter_prots.fasta -d r_0/db/Pool_new_unclustered -o r_0/Contigs_prots_vs_New_unclustered.tab -a 8 -m 8 -e 0.001 >> /root/log_out 2>> /root/log_err Step 2 : /usr/local/bin/Scripts/Step_2_merge_contigs_annotation.pl /root/fasta/VIRSorter_mga_final.predict Contigs_prots_vs_Phage_Gene_Catalog.tab Contigs_prots_vs_Phage_Gene_unclustered.tab Contigs_prots_vs_PFAMa.tab Contigs_prots_vs_PFAMb.tab /data/Phage_gene_catalog_plus_viromes/Phage_Clusters_current.tab VIRSorter_affi-contigs.csv >> /root/log_out 2>> /root/log_err Step 3 : /usr/local/bin/Scripts/Step_3_highlight_phage_signal.pl VIRSorter_affi-contigs.csv VIRSorter_phage-signal.csv >> /root/log_out 2>> /root/log_err Setting up the final result file Step 4 : /usr/local/bin/Scripts/Step_4_summarize_phage_signal.pl VIRSorter_affi-contigs.csv VIRSorter_phage-signal.csv VIRSorter_global-phage-signal.csv VIRSorter_new_prot_list.csv >> /root/log_out 2>> /root/log_err Step 5 : /usr/local/bin/Scripts/Step_5_get_phage_fasta-gb.pl VIRSorter >> /root/log_out 2>> /root/log_err Cleaning the output directory rm -r r_0/db : mv: cannot stat 'formatdb.log': No such file or directory

Thanks in advance for your help.

Cheers

Daniele

[simroux]

Hi Daniele,

Could you paste the content of "log_out" and "log_err" please ? Also, it looks like VirSorter is trying to work in the "/root/" directory, do you know if maybe it may not have permissions to create and write files there ?

[DanieleDeco]

Hi, Simroux

You are right I don't have the permission to write and access the root. At the moment I cannot send you the "log_out" and "log_err" files, I have to wait for the administrator on Monday. Can I force VirSorter to work in another directory?

Cheers

Daniele

[simroux]

Hi Daniele, Don't worry about the log_out and log_err, if you indeed did not have the rights permissions, these are most likely empty.

You can change the directory into which VirSorter is running by using the option "--wdir"

[DanieleDeco]

Hi Simroux,

I changed the directory and I modified my permissions. But I get the same error and no output. What could that be?

Cheers

Daniele

drwxrwxrwx. 4 bioresource bioresource 4096 May 27 18:01 SPAdes-3.10.1-Linux/ -rwxrwxrwx. 1 bioresource bioresource 10294085 May 27 18:01 SPAdes-3.10.1-Linux.tar.gz drwxrwxrwx. 4 bioresource bioresource 4096 Aug 31 11:24 Trimmed/ -rwxrwxrwx. 1 bioresource docker 86825 Sep 24 16:02 Virome_Miseq_samples.xlsx drwxrwxrwx. 5 bioresource docker 4096 Sep 23 14:08 Virsorter

[bioresource@bioresorce1 Virome]$ docker run -v /home/bioresource/media/bay1/daniele/Virome/Virsorter/virsorter-data:/data -v /home/bioresource/media/bay1/daniele/Virome/Virsorter/virsorter-run:/wdir --rm discoenv/virsorter:v1.0.3 --db 2 --fna /wdir/contigs.fasta --wdir /home/bioresource/media/bay1/daniele/Virome/Virsorter/ Dataset : VIRSorter Fna file : /wdir/contigs.fasta Db : 2 Wdir : /home/bioresource/media/bay1/daniele/Virome/Virsorter/ Custom phages:

%#%#%#%#%# Processing VIRSorter....

Step 0.5 : /usr/local/bin/Scripts/Step_1_contigs_cleaning_and_gene_prediction.pl VIRSorter /home/bioresource/media/bay1/daniele/Virome/Virsorter/fasta /home/bioresource/media/bay1/daniele/Virome/Virsorter/fasta/input_sequences.fna 2 >> /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_out 2>> /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_err

Step 0.8 : /usr/local/bin/hmmsearch --tblout Contigs_prots_vs_PFAMa.tab --cpu 8 -o Contigs_prots_vs_PFAMa.out --noali /data/PFAM_27/Pfam-A.hmm /home/bioresource/media/bay1/daniele/Virome/Virsorter/fasta/VIRSorter_prots.fasta >> /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_out 2>> /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_err

Step 0.9 : /usr/local/bin/hmmsearch --tblout Contigs_prots_vs_PFAMb.tab --cpu 8 -o Contigs_prots_vs_PFAMb.out --noali /data/PFAM_27/Pfam-B.hmm /home/bioresource/media/bay1/daniele/Virome/Virsorter/fasta/VIRSorter_prots.fasta >> /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_out 2>> /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_err

Revision 0

Out : Step 1.2 : /usr/local/bin/hmmsearch --tblout r_0/Contigs_prots_vs_New_clusters.tab --cpu 8 -o r_0/Contigs_prots_vs_New_clusters.out --noali r_0/db/Pool_clusters.hmm /home/bioresource/media/bay1/daniele/Virome/Virsorter/fasta/VIRSorter_prots.fasta >> /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_out 2>> /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_err

Step 1.3 : /usr/local/bin/blastall -p blastp -i /home/bioresource/media/bay1/daniele/Virome/Virsorter/fasta/VIRSorter_prots.fasta -d r_0/db/Pool_new_unclustered -o r_0/Contigs_prots_vs_New_unclustered.tab -a 8 -m 8 -e 0.001 >> /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_out 2>> /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_err

Step 2 : /usr/local/bin/Scripts/Step_2_merge_contigs_annotation.pl /home/bioresource/media/bay1/daniele/Virome/Virsorter/fasta/VIRSorter_mga_final.predict Contigs_prots_vs_Phage_Gene_Catalog.tab Contigs_prots_vs_Phage_Gene_unclustered.tab Contigs_prots_vs_PFAMa.tab Contigs_prots_vs_PFAMb.tab /data/Phage_gene_catalog_plus_viromes/Phage_Clusters_current.tab VIRSorter_affi-contigs.csv >> /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_out 2>> /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_err

Step 3 : /usr/local/bin/Scripts/Step_3_highlight_phage_signal.pl VIRSorter_affi-contigs.csv VIRSorter_phage-signal.csv >> /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_out 2>> /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_err

Setting up the final result file Step 4 : /usr/local/bin/Scripts/Step_4_summarize_phage_signal.pl VIRSorter_affi-contigs.csv VIRSorter_phage-signal.csv VIRSorter_global-phage-signal.csv VIRSorter_new_prot_list.csv >> /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_out 2>> /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_err

Step 5 : /usr/local/bin/Scripts/Step_5_get_phage_fasta-gb.pl VIRSorter >> /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_out 2>> /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_err

Cleaning the output directory rm -r r_0/db : mv: cannot stat 'fasta/': No such file or directory mv: cannot stat 'formatdb.log': No such file or directory mv: cannot stat 'log_err': No such file or directory mv: cannot stat 'log_out': No such file or directory

[simroux]

Hi Daniele, Do you have anything in /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_err and /home/bioresource/media/bay1/daniele/Virome/Virsorter/log_out, or are these files empty / missing ?

[DanieleDeco]

Hi simroux, I don't have any file, all of them are missing.

Cheers

Daniele

[simroux]

Hi Daniele, This is very surprising, because it looks like a permission issue again. The only other explanation I could see is that VirSorter doesn't see the input fasta file.

In the command "docker run -v /home/bioresource/media/bay1/daniele/Virome/Virsorter/virsorter-data:/data -v /home/bioresource/media/bay1/daniele/Virome/Virsorter/virsorter-run:/wdir --rm discoenv/virsorter:v1.0.3 --db 2 --fna /wdir/contigs.fasta --wdir /home/bioresource/media/bay1/daniele/Virome/Virsorter/", could you try changing "/wdir/contigs.fasta" by the full path of your input file ?

[DanieleDeco]

Hi Simroux,

I got these output files (pic), but I didn't get the other ones.

rm -r r_0/db : mv: cannot stat 'fasta/': No such file or directory mv: cannot stat 'formatdb.log': No such file or directory mv: cannot stat 'log_err': No such file or directory mv: cannot stat 'log_out': No such file or directory

That's strange.

Cheers

[simroux]

Hi Daniele, This is unusual.. if you don't mind, could you create a zip archives with this output directory and your input file and send it over (you can send these through email if you don't want to post it publicly). Thanks, Cheers,

Simon

[DanieleDeco]

Hi Simroux, Please find attached the output directory in a zip file. Archive.zip

Thanks again for your help. Cheers

Daniele

[simroux]

Hi Daniele, Some good news looking at these files, VirSorter pipeline was almost entirely done before the errors you see in the logs kick in. This means that VirSorter results are actually available (they are in the file "VIRSorter_global-phage-signal.csv"). The step after this one (where VirSorter is supposed to generated corresponding genbank files for these sequences and then re-arrange the files in nicely organized folders) is where things fail, however I can't seem to understand why at the moment.

I'll keep looking into this and try to replicate the error, but in the meantime you can use these results, and you can also upload your input file on CyVerse (https://de.cyverse.org/de/) and run VirSorter there to make sure that you get identical results.

[DanieleDeco]

Hi Simrioux, The online version is working pretty fine. I think that from now on I am going to use that one. I have just one more question, I would like to know what the files nett_filtered.fasta are (in the fasta_files folder) . Thanks for your help. Daniele

[simroux]

Hi Daniele, This fasta file includes all the contigs that VirSorter will consider, i.e. all contigs with at least 2 predicted ORFs. It also include the "circularized" version of contigs for which identical 5' and 3' ends were detected (i.e. removal of the matching ends).

Best, Simon

[chadsmith123]

I had this problem, docker was writing the data as root. Try

docker run -u id -u:id -g... to set the user and group to current user.