kraken in metAMOS v1.5rc3 fails during annotate test phase

[aldembert]

Can I get a clue/guide please on interpreting what went wrong with MetAmos while running the run_pipeline_test.sh in the kraken step?. Everything went fine during installation but Im getting the following errors when testing the installation of metAMOS:

$ sudo ./run_pipeline_test.sh Project directory already exists, please specify another Alternatively, use runPipeline to run an existing project

Starting Task = runpipeline.RUNPIPELINE Starting metAMOS pipeline Warning: Celera Assembler is not found, some functionality will not be available Warning: BLASR is not found, some functionality will not be available Warning: Newbler is not found, some functionality will not be available Warning: ktImportTaxonomy is not found, some functionality will not be available Warning: MetaGeneMark is not found, some functionality will not be available Warning: Prokka is not found, some functionality will not be available Warning: SignalP+ is not found, some functionality will not be available Warning: REAPR is not found, some functionality will not be available Warning: FRCbam is not found, some functionality will not be available [Available RAM: 5 GB] [ There is 5 GB of RAM currently available on this machine, suggested minimum of 32 GB [ Enabling low MEM mode, might slow down some steps in pipeline [ If more RAM is available than what is listed above, please close down other programs and restart runPipeline [Available CPUs: 4] [ Only 4 CPU cores available, some modules might take awhile to complete [ *Disabling all BLAST (where possible)

---

Tasks which will be run:

Task = assemble.SplitAssemblers Task = assemble.Assemble Task = assemble.CheckAsmResults Task = assemble.SplitMappers Task = mapreads.MapReads Task = mapreads.CheckMapResults Task = mapreads.SplitForORFs Task = findorfs.FindORFS Task = validate.Validate Task = findreps.FindRepeats Task = annotate.Annotate Task = fannotate.FunctionalAnnotation Task = scaffold.Scaffold Task = findscforfs.FindScaffoldORFS Task = abundance.Abundance Task = propagate.Propagate Task = classify.Classify Task = postprocess.Postprocess

---

metAMOS configuration summary: metAMOS Version: v1.5rc2 "Praline Brownie" workflows: core,optional,imetamos Time and Date: 2014-11-27 Working directory: /home/user/apps/metAMOS-1.5rc3/Test/test1 Prefix: proba K-Mer: 55 Threads: 15 Taxonomic level: class Verbose: False Steps to skip: MultiAlign, FunctionalAnnotation, FindRepeats Steps to force: Abundance, FindORFS, Annotate, Propagate, MapReads, Assemble, FindScaffoldORFS, Classify

[citation] MetAMOS Treangen, TJ ⇔ Koren, S, Sommer, DD, Liu, B, Astrovskaya, I, Ondov, B, Darling AE, Phillippy AM, Pop, M. MetAMOS: a modular and open source metagenomic assembly and analysis pipeline. Genome biology, 14(1), R2, 2013.

Step-specific configuration: [abundance] MetaPhyler /home/user/apps/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64 Liu B, Gibbons T, Ghodsi M, Treangen T, Pop M. Accurate and fast estimation of taxonomic profiles from metagenomic shotgun sequences. BMC Genomics. 2011;12 Suppl 2:S4. Epub 2011 Jul 27.

[multialign] M-GCAT /home/user/apps/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64 Treangen TJ, Messeguer X. M-GCAT: interactively and efficiently constructing large-scale multiple genome comparison frameworks in closely related species. BMC Bioinformatics, 2006.

[fannotate] BLAST /home/user/apps/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64 Altschul SF, Gish W, Miller W, Myers EW, Lipman DJ. Basic local alignment search tool. J Mol Biol. 1990 Oct 5;215(3):403-10.

[scaffold] Bambus 2 /home/user/apps/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin Koren S, Treangen TJ, Pop M. Bambus 2: scaffolding metagenomes. Bioinformatics 27(21): 2964-2971 2011.

[findorfs] FragGeneScan /home/user/apps/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64 Rho M, Tang H, Ye Y: FragGeneScan: predicting genes in short and error-prone reads. Nucleic Acids Research 2010, 38:e191-e191.

[annotate] Kraken /home/user/apps/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/kraken/bin Wood DE, Salzberg SL: Kraken: ultrafast metagenomic sequence classification using exact alignments. Genome Biology 2014, 15:R46.

[assemble] SOAPdenovo /home/user/apps/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64 Li Y, Hu Y, Bolund L, Wang J: State of the art de novo assembly of human genomes from massively parallel sequencing data.Human genomics 2010, 4:271-277.

[mapreads] Bowtie /home/user/apps/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64 Langmead B, Trapnell C, Pop M, Salzberg SL. Ultrafast and memory-efficient alignment of short DNA sequences to the human genome. Genome Biol. 2009;10(3):R25. Epub 2009 Mar 4.

[preprocess] metAMOS built-in filtering N/A

[validate] LAP /home/user/apps/metAMOS-1.5rc3/LAP Ghodsi M, Hill CM, Astrovskaya I, Lin H, Sommer DD, Koren S, Pop M. De novo likelihood-based measures for comparing genome assemblies. BMC research notes 6:334, 2013.

[other] Krona

Ondov BD, Bergman NH, Phillippy AM. Interactive metagenomic visualization in a Web browser. BMC Bioinformatics. 2011 Sep 30;12:385.

sh: 1: Syntax error: Bad fd number Starting Task = assemble.ASSEMBLE Job = [preprocess.success -> *.run] completed Completed Task = assemble.SplitAssemblers Job = [soapdenovo.55.run -> soapdenovo.55.asm.contig] completed Completed Task = assemble.Assemble Job = [[soapdenovo.55.asm.contig] -> [assemble.ok]] completed Completed Task = assemble.CheckAsmResults Uptodate Task = assemble.SplitMappers

Starting Task = mapreads.MAPREADS Job = [soapdenovo.55.asm.contig -> soapdenovo.55.contig.cvg] completed Completed Task = mapreads.MapReads Job = [[soapdenovo.55.contig.cvg] -> [mapreads.ok]] completed Completed Task = mapreads.CheckMapResults Uptodate Task = mapreads.SplitForORFs Starting Task = findorfs.FINDORFS Job = [soapdenovo.55.contig.cvg -> soapdenovo.55.faa] completed Completed Task = findorfs.FindORFS Starting Task = validate.VALIDATE Job = [[soapdenovo.55.faa] -> [validate.ok]] completed Completed Task = validate.Validate Starting Task = findrepeats.FINDREPEATS Job = [proba.fna -> proba.repeats] completed Completed Task = findreps.FindRepeats Starting Task = annotate.ANNOTATE [***** ****ERROR****** During annotate, the following command failed with return code 2:

/home/user/apps/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/kraken/bin/kraken --threads 15 -db /home/user/apps/metAMOS-1.5rc3/Utilities/DB//kraken /home/user/apps/metAMOS-1.5rc3/Test/test1/Annotate/in/proba.asm.contig --preload | kraken-filter --db /home/user/apps/metAMOS-1.5rc3/Utilities/DB//kraken --threshold 0.05 > /home/user/apps/metAMOS-1.5rc3/Test/test1/Annotate/out/proba.hits

****DETAILS****** Last 10 commands run before the error (/home/user/apps/metAMOS-1.5rc3/Test/test1/Logs/COMMANDS.log) |2014-11-27 08:15:35| touch /home/user/apps/metAMOS-1.5rc3/Test/test1/Logs/findrepeats.skip |2014-11-27 08:15:36|# [ANNOTATE] |2014-11-27 08:15:36| touch /home/user/apps/metAMOS-1.5rc3/Test/test1/Annotate/out/proba.annots |2014-11-27 08:15:36| unlink /home/user/apps/metAMOS-1.5rc3/Test/test1/Annotate/in/proba.asm.contig |2014-11-27 08:15:36| ln /home/user/apps/metAMOS-1.5rc3/Test/test1/Assemble/out/proba.asm.contig /home/user/apps/metAMOS-1.5rc3/Test/test1/Annotate/in/proba.asm.contig |2014-11-27 08:15:36| unlink /home/user/apps/metAMOS-1.5rc3/Test/test1/Annotate/out/proba.hits |2014-11-27 08:15:36| rm -f /home/user/apps/metAMOS-1.5rc3/Test/test1/Annotate/out/.hits |2014-11-27 08:15:36| rm -f /home/user/apps/metAMOS-1.5rc3/Test/test1/Annotate/out/.epsilon-nb_results.txt |2014-11-27 08:15:36| rm -f /home/user/apps/metAMOS-1.5rc3/Test/test1/Annotate/out/*.phymm.out |2014-11-27 08:15:37| /home/user/apps/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/kraken/bin/kraken --threads 15 -db /home/user/apps/metAMOS-1.5rc3/Utilities/DB//kraken /home/user/apps/metAMOS-1.5rc3/Test/test1/Annotate/in/proba.asm.contig --preload | kraken-filter --db /home/user/apps/metAMOS-1.5rc3/Utilities/DB//kraken --threshold 0.05 > /home/user/apps/metAMOS-1.5rc3/Test/test1/Annotate/out/proba.hits

Last 10 lines of output (/home/user/apps/metAMOS-1.5rc3/Test/test1/Logs/ANNOTATE.log) unlink: no se puede deshacer el enlace «/home/user/apps/metAMOS-1.5rc3/Test/test1/Annotate/out/proba.hits»: No existe el archivo o el directorio kraken-filter: can't open nodes file: No existe el archivo o el directorio classify: error opening /home/user/apps/metAMOS-1.5rc3/Utilities/DB//kraken/taxonomy/nodes.dmp: No such file or directory

Please veryify input data and restart MetAMOS. If the problem persists please contact the MetAMOS development team. ****ERROR******

---

rm: no se puede borrar «/home/user/apps/metAMOS-1.5rc3/Test/test1/Logs/annotate.ok»: No existe el archivo o el directorio

Oops, MetAMOS finished with errors! see text in red above for details.

Thanks for your attention.

[skoren]

From your log and the missing programs listed at the beginning, it looks like INSTALL.py did not run successfully. The Kraken database is missing, which is why your run is failing. Does your system limit internet connections and did INSTALL.py report any errors while running? If you continue to have trouble using INSTALL.py, you can try the frozen binary instead. However, the resources available on your system (5GB, 4cores) will mean you will be unable to run most of the tools included within metAMOS. I would suggest installing on a machine with at least 64GB of ram. For more information, see the hardware documentation listed on our help pages: http://metamos.readthedocs.org/en/v1.5rc3/content/hardware.html

[aldembert]

Hi,

Thanks for the so quick response Sergey, I do not recall seeing any error relating the Kraken database, the only related issue is that the installer showed that there was not enough memory so it would install a minimal version of the kraken DB. Regarding the frozen binary could you tellme where I can find since I do have access to a larger computer with 256Gb of RAM and 64 cores but this machine currently doesnt have internet access so I would like to try metAMOS with this frozen binary you talk about.

Thanks a lot and Im looking forward to your response.

Salvador Ángel

On Fri, Nov 28, 2014 at 11:37 AM, Sergey Koren notifications@github.com wrote:

From your log and the missing programs listed at the beginning, it looks like INSTALL.py did not run successfully. The Kraken database is missing, which is why your run is failing. Does your system limit internet connections and did INSTALL.py report any errors while running? If you continue to have trouble using INSTALL.py, you can try the frozen binary instead. However, the resources available on your system (5GB, 4cores) will mean you will be unable to run most of the tools included within metAMOS. I would suggest installing on a machine with at least 64GB of ram. For more information, see the hardware documentation listed on our help pages: http://metamos.readthedocs.org/en/v1.5rc3/content/hardware.html

— Reply to this email directly or view it on GitHub https://github.com/marbl/metAMOS/issues/171#issuecomment-64916325.

[skoren]

I verified that the Kraken miniature DB is working as expected on a new installation of metAMOS. You can try removing Utilities/DB/kraken and re-running INSTALL.py to re-download the DB in case it was not downloaded properly before. All the metAMOS documentation, including the frozen binary is on readthedocs: http://metamos.readthedocs.org/en/v1.5rc3/ The frozen binary is: http://metamos.readthedocs.org/en/v1.5rc3/content/frozenbinary.html

[aldembert]

Got it, Thanks Sergey¡

I will try both solutions right away¡.

Thanks again for your time and Im looking forward to reach you again with hopefully more constructive comments about metAMOS not support-related.

[aldembert]

Dear Sergey,

I just finished trying your solution to mi first inquiry, the one related to the proble with the execution of Kraken during the Annotate step of the Test process, and I happily saw that you were right about me lackinf Kraken DB files, for some reason I didn´t have the taxonomy folder and other Kraken related files, so I proceeded as you told me and reinstall deleting first the DB folder of kraken. Nonetheless after reinstalling and running again the test process I faced another error: This time it was some Krona related error. Im Since I didn´t want to write you again without anything constructive, as I said in my last E-mail, I tried to work around the problem by my self and I tried to reinstall the Krona perl module included in metAMOS, but when I tried to do it I got the following error:

---

sudo perl install.pl [ 1:27]

syntax error at install.pl line 58, near ")"

(Might be a runaway multi-line () string starting on line 37)

syntax error at install.pl line 80, near "}"

Illegal declaration of subroutine main::createDir at install.pl line 89.

---

After searching the metAMOS issues I found Issue 163 ( https://github.com/marbl/metAMOS/issues/163) but the lines didn´t coincide with the ones you mentioned that had to be corrected (just by a tiny difference) since it said that the "qw" started in line 37 but in the install perl file that I got with metAMOS 1.5rc3 the qw actually started in line 36. Any way, long story short I added the parenthesis that was suggested in the issue 163 and that fixed the problem of the module installation so I ran again the metAMOS installation and the test processes and this time everything went OK. Im showing below the final successful output. Anyway I just wanted inform you about this Krona issue in my installation since you also took time to check my problem and may be this could help you in future issues. By the way I saw in the 163 issue that the respective error was repaired in a recent branch but Im not sure that the patch solved all the cases of the problem since the lines in the install file for the krona module that I have have the small difference that I just described.

Regarding the issue of the frozen binaries installation in my large server system I going to begin to deal with the manual installation of the perl modules since some of them are lacking and as I told you I dont have Internet in that server machine yet. I will let you know how this works.

Thanks a lot for your time.

[skoren]

This looks like a perl version incompatibility with KronaTools 2.2/2.3. The latest checkout of version 1.5rc3 uses KronaTools 2.4 which fixes the issue. If you download and install KronaTools-2.4, it should fix your error. You should also be able to change line 124 in INSTALL.py to be:

kronaTools = "KronaTools-2.4"

remove KronaTools from your metAMOS directory and re-run INSTALL.py to fix the issue.

[skoren]

Closing for inactivity, please re-open if the suggestion above does not fix your issue.