nmquijada / tormes

Making whole bacterial genome sequencing data analysis easy
GNU General Public License v3.0
99 stars 32 forks source link

error with serotypefinder #20

Closed valery-shap closed 4 years ago

valery-shap commented 4 years ago

Hello, Thank you for the tool, it might be very useful for me. I have some errors during working, could you comment please. {'serotypefinder': {'results': {'H_type': {'fliC_90_AY250023_H45': {'HSP_length': 1707, 'accession': 'AY250023', 'contig_name': 'NODE_32_length_68586_cov_83.397099', 'coverage': 100.0, 'gene': 'fliC', 'hit_id': 'NODE_32_length_68586_cov_83.397099:59707..61413:fliC_90_AY250023_H45:98.301113', 'identity': 98.3, 'position_in_ref': '1..1707', 'positions_in_contig': '59707..61413', 'serotype': 'H45', 'template_length': 1707}}, 'O_type': {'wzx_72_DQ069297_O66': {'HSP_length': 1338, 'accession': 'DQ069297', 'contig_name': 'NODE_94_length_2903_cov_24.776657', 'coverage': 100.0, 'gene': 'wzx', 'hit_id': 'NODE_94_length_2903_cov_24.776657:293..1630:wzx_72_DQ069297_O66:99.925262', 'identity': 99.93, 'position_in_ref': '1..1338', 'positions_in_contig': '293..1630', 'serotype': 'O66', 'template_length': 1338}, 'wzy_O59_DQ069297_O66': {'HSP_length': 923, 'accession': 'DQ069297', 'contig_name': 'NODE_89_length_3724_cov_21.140117', 'coverage': 72.92, 'gene': 'wzy', 'hit_id': 'NODE_89_length_3724_cov_21.140117:1..923:wzy_O59_DQ069297_O66:72.763605', 'identity': 99.78, 'position_in_ref': '49..969', 'positions_in_contig': '1..923', 'serotype': 'O66', 'template_length': 1263}}}, 'run_info': {'date': '21.04.2020', 'time': '22:01:04'}, 'user_input': {'file_format': 'fasta', 'filename(s)': ['/home/valery/bac11_rep/genomes/Sample1.fasta'], 'method': 'blast'}}} Can't locate Bio/SeqIO.pm in @INC (you may need to install the Bio::SeqIO module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at /home/valery/miniconda3/envs/tormes-1.1/bin/../CGE-modules/fimtyper/fimtyper.pl line 9. BEGIN failed--compilation aborted at /home/valery/miniconda3/envs/tormes-1.1/bin/../CGE-modules/fimtyper/fimtyper.pl line 9. sed: can't read /home/valery/bac11_rep/fimH_typing/Sample1/results_tab.txt: No such file or directory

Warning message: package ‘rmarkdown’ was built under R version 3.6.2

processing file: tormes_report.Rmd |. | 2% ordinary text without R code

|... | 4% label: unnamed-chunk-1 (with options) List of 3 $ echo : logi FALSE $ message: logi FALSE $ warning: logi FALSE

Warning messages: 1: package 'ggplot2' was built under R version 3.6.2 2: package 'knitr' was built under R version 3.6.2 3: package 'plotly' was built under R version 3.6.3 4: package 'reshape2' was built under R version 3.6.3

nmquijada commented 4 years ago

Hi @valery-shap The error appears with fimtyper as it is unable to locate the Bio/SeqIO.pm perl module. You can try to install that module, after activating tormes environment, by using conda (conda install -c bioconda perl-bioperl) or by using cpan (cpan Bio::SeqIO). Please, try to do so and re-run the analysis. Let me know how it goes! Narciso

valery-shap commented 4 years ago

Thank you very much for your reply. I tried the first command and the second, and there was no result. the same error. then i decided to update the env and seems that nothing works now.

Parse error at line 821: missing colon in auxiliary data

== Error == system call for: "['/home/valery/miniconda3/envs/tormes-1.1/bin/spades-corrector-core', '/home/valery/bac11/assembly/Sample1_assembly/mismatch_corrector/contigs/configs/corrector.info', '/home/valery/bac11/assembly/Sample1_assembly/misc/assembled_contigs.fasta']" finished abnormally, err code: -6

and then a lot of errors.

cp: cannot stat '/home/valery/bac11/antibiotic_resistance_genes//tab': No such file or directory cp: cannot stat '/home/valery/bac11/virulence_genes/tab': No such file or directory cp: cannot stat '/home/valery/bac11/point_mutations//_PointFinder_results.txt': No such file or directory ERROR: Can't open '/home/valery/bac11/report_files/resfinder_min90.tab' to summarize. ERROR: Can't open '/home/valery/bac11/report_files/card_min90.tab' to summarize. ERROR: Can't open '/home/valery/bac11/report_files/argannot_min90.tab' to summarize. cut: '/home/valery/bac11/report_files/resfinder_min90.tab': No such file or directory cut: /home/valery/bac11/report_files/tmp2.txt: No such file or directory paste: /home/valery/bac11/report_files/tmp2.txt: No such file or directory cp: cannot stat '/home/valery/bac11/fimH_typing/fimH_typing.txt': No such file or directory cp: cannot stat '/home/valery/bac11/serotyping/serotyping.txt': No such file or directory cp: cannot stat '/home/valery/bac11/plasmids/tab': No such file or directory Warning message: package ‘rmarkdown’ was built under R version 3.6.2

so looks like i have to remove the env.

what is better way to install everything one more time?

1.conda env create -n tormes-1.1 --file tormes-1.1.yml 2.conda activate tormes-1.1

3.tormes-setup

OR before setup

quast-download-gridss quast-download-silva quast-download-busco conda install -c bioconda perl-bioperl

and then tormes-setup

?

valery-shap commented 4 years ago

Error with busco database

Downloading BUSCO database... Downloading bacteria database (file: bacteria.tar.gz)...

ERROR! Failed downloading bacteria database (url: https://busco.ezlab.org/v2/datasets/bacteria_odb9.tar.gz), QUAST functionality will be limited! Exception caught: HTTP Error 301: Moved Permanently You can try to download the file manually, place it in /home/valery/miniconda3/envs/tormes-1.1/lib/python3.6/site-packages/quast_libs/busco/bacteria.tar.gz and restart QUAST WARNING: Failed to download bacteria database from https://busco.ezlab.org/v2/datasets/bacteria_odb9.tar.gz and unpack it into /home/valery/miniconda3/envs/tormes-1.1/lib/python3.6/site-packages/quast_libs/busco Downloading BUSCO database... Downloading eukaryota database (file: eukaryota.tar.gz)...

ERROR! Failed downloading eukaryota database (url: https://busco.ezlab.org/v2/datasets/eukaryota_odb9.tar.gz), QUAST functionality will be limited! Exception caught: HTTP Error 301: Moved Permanently You can try to download the file manually, place it in /home/valery/miniconda3/envs/tormes-1.1/lib/python3.6/site-packages/quast_libs/busco/eukaryota.tar.gz and restart QUAST WARNING: Failed to download eukaryota database from https://busco.ezlab.org/v2/datasets/eukaryota_odb9.tar.gz and unpack it into /home/valery/miniconda3/envs/tormes-1.1/lib/python3.6/site-packages/quast_libs/busco Downloading BUSCO database... Downloading fungi database (file: fungi.tar.gz)...

ERROR! Failed downloading fungi database (url: https://busco.ezlab.org/v2/datasets/fungi_odb9.tar.gz), QUAST functionality will be limited! Exception caught: HTTP Error 301: Moved Permanently You can try to download the file manually, place it in /home/valery/miniconda3/envs/tormes-1.1/lib/python3.6/site-packages/quast_libs/busco/fungi.tar.gz and restart QUAST WARNING: Failed to download fungi database from https://busco.ezlab.org/v2/datasets/fungi_odb9.tar.gz and unpack it into /home/valery/miniconda3/envs/tormes-1.1/lib/python3.6/site-packages/quast_libs/busco

nmquijada commented 4 years ago

Hi @valery-shap In this last comment, your issue is with the databases from QUAST that cause a warning when downloading from conda. However, QUAST is used in TORMES for the calculation of assembly/genomes statistics. The lack of those databases do not interfere with its work within TORMES. The message before that showed a problem during assembly with SPAdes. The rest of the issues it returns were due to the lack of already assembled genomes (check the tormes.log file to see possible warnings).

Let's start from the beginning. I assume you are using tormes version 1.1. First remove your current tormes environment: conda enve remove -n <NAME OF YOUR ENVIRONMENT>

Then, install tormes 1.1:

wget https://anaconda.org/nmquijada/tormes-1.1/2020.04.08.105015/download/tormes-1.1.yml
conda env create -n tormes-1.1 --file tormes-1.1.yml
conda activate tormes-1.1
tormes-setup

Now, try to run your analysis again, but redirect the screen output to a file called "error-tormes.txt". For instance: tormes --metadata YOUR-METADATA.txt --output YOUR-OUTPUT <other opions> &>>error-tormes.txt 2>>error-tormes.txt & By doing this, tormes will run in the backgroun of your machine and you won't see anything, but it will be running. So please don't turn down the system. You can check your computer tasks with top or ps -fu <your USER>. Also check regularly the tormes.log file. When the analysis is finished, please share that error-tormes.txt file with me.

In order to anticipate possible problems due to Bioperl, could you please post the output of:

echo $PERL5LIB
echo $PERL_LOCAL_LIB_ROOT

Thanks, Narciso

valery-shap commented 4 years ago

the error with spades was because lack of memory, i found the answer here: https://github.com/ablab/spades/issues/210

output of installing bioperl:

(tormes-1.1) valery@WS-SN010009:~$ conda install -c bioconda perl-bioperl Collecting package metadata (current_repodata.json): done Solving environment: done

==> WARNING: A newer version of conda exists. <== current version: 4.8.2 latest version: 4.8.3

Please update conda by running

$ conda update -n base -c defaults conda

All requested packages already installed.

(tormes-1.1) valery@WS-SN010009:~$ cpan Bio::SeqIO Reading '/home/valery/.cpan/Metadata' Database was generated on Thu, 23 Apr 2020 08:17:03 GMT Fetching with LWP: http://www.cpan.org/authors/01mailrc.txt.gz Reading '/home/valery/.cpan/sources/authors/01mailrc.txt.gz' ............................................................................DONE Fetching with LWP: http://www.cpan.org/modules/02packages.details.txt.gz Reading '/home/valery/.cpan/sources/modules/02packages.details.txt.gz' Database was generated on Fri, 24 Apr 2020 06:29:03 GMT ............. New CPAN.pm version (v2.27) available. [Currently running version is v2.18] You might want to try install CPAN reload cpan to both upgrade CPAN.pm and run the new version without leaving the current session.

...............................................................DONE Fetching with LWP: http://www.cpan.org/modules/03modlist.data.gz Reading '/home/valery/.cpan/sources/modules/03modlist.data.gz' DONE Writing /home/valery/.cpan/Metadata Bio::SeqIO is up to date (1.7.7).

the output of echo: (tormes-1.1) valery@WS-SN010009:~$ echo $PERL5LIB

(tormes-1.1) valery@WS-SN010009:~$ echo $PERL_LOCAL_LIB_ROOT

(tormes-1.1) valery@WS-SN010009:~$

and the output of tormes still with error Can't locate Bio/SeqIO.pm in @INC (you may need to install the Bio::SeqIO module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at /home/valery/miniconda3/envs/tormes-1.1/bin/../CGE-modules/fimtyper/fimtyper.pl line 9. BEGIN failed--compilation aborted at /home/valery/miniconda3/envs/tormes-1.1/bin/../CGE-modules/fimtyper/fimtyper.pl line 9. sed: can't read /home/valery/bac11/fimH_typing/Sample1/results_tab.txt: No such file or directory

and i attached file eroor-tormes.txt error-tormes.txt

nmquijada commented 4 years ago

Hi @valery-shap Thanks for sharing the file. OK, I see. TORMES finished the analysis but you are missing the fimhtyper output as that software is not being called properly. I am afraid this is an issue about using the system perl instead of the perl in TORMES environment. In order to check that, can you try the following steps? (activate tormes environment first)

conda install -c bioconda perl-try-tiny-retry
sed "s/\$FIMTYPER -d/perl \$FIMTYPER -d/" /home/valery/miniconda3/envs/tormes-1.1/bin/tormes > /home/valery/miniconda3/envs/tormes-1.1/bin/tormes-bugfix
chmod 775 home/valery/miniconda3/envs/tormes-1.1/bin/tormes-bugfix

Then, run your analysis again, by using "tormes-bugfix" instead of "tormes" (tormes-bugfix --metadata YOUR-METADATA.txt --output YOUR-OUTPUT <other opions>)

Let me know if it works. Narciso

valery-shap commented 4 years ago

Thank you very much! Everything works excellent.

Valery