search

marbl / metAMOS

A metagenomic and isolate assembly and analysis pipeline built with AMOS
http://marbl.github.io/metAMOS
Other
93 stars 45 forks source link

Test suite errors. #180

Closed JustGitting closed 9 years ago

JustGitting commented 9 years ago

Hi everyone,

I've been trying to run the test suite after installing metAMOS from the git clone of 1.5rc3.

$ git clone --branch "v1.5rc3" https://github.com/marbl/metAMOS.git
$ mv metAMOS/ metAMOS-1.5rc3_git

I had to run the INSTALL script a few times before it completed successfully.

$ export http_proxy="http://proxy.abc.com:8080"
$ export ftp_proxy="ftp://proxy.abc.com:8080"
$ python INSTALL.py core imetamos optional

I've run the check installation script with the following results.

$ python  ./check_install.py ./ allfiles.txt
Installation in ./ looks ok, have fun!

However, using the "required files list" fails, and the ".//Utilities/cpp/Linux-x86_64/MetaVelvet/scripts" does not even exist.

$ python  ./check_install.py ./ required_file_list.txt
ERROR: Required file .//Utilities/cpp/Linux-x86_64/MetaVelvet/scripts/shuffleSequences_fasta.pl not found! Please reinstall MetAMOS

I've tried reinstalling MetAMOS, but it does not have any errors:

$ python INSTALL.py core imetamos optional
<<Welcome to metAMOS install>>
Selected to install workflowName: CORE.
Selected to install workflowName: OPTIONAL.
Selected to install workflowName: IMETAMOS.
Will automatically install:
        Pysam
        Fastqc
        Ca
        Ale
        Sga
        Velvet-Sc
        Metaphyler
        Phylosift
        Metavelvet
        Velvet
        Soap2
        Frcbam
        Quast
        Eautils
        Ray
        Numpy
        Freebayes
        Fcp
        Edena
        Uniprot
        Cython
        Cgal
        Mira
        Matplotlib
        Prokka
        Idba
        Cmake
        Lap
        Masurca
        Kronatools
        Abyss
        Sra
        Spades
        Kraken
        Psutil
        Setuptools
        Amos
        Phmmer
        Kmergenie
        Reapr
Found jellyfish v2.1.3
Kraken requires jellyfish version 1
Checking whether refseq_protein is complete. Expecting 17 partitions.

Run setup.py..
running install_scripts
running egg_info
writing metAMOS.egg-info/PKG-INFO
writing top-level names to metAMOS.egg-info/top_level.txt
writing dependency_links to metAMOS.egg-info/dependency_links.txt
reading manifest file 'metAMOS.egg-info/SOURCES.txt'
writing manifest file 'metAMOS.egg-info/SOURCES.txt'
running build_scripts
copying build/scripts-2.7/runPipeline.py -> /group/metagen/dbtools/src/metAMOS-1.5rc3_git
copying build/scripts-2.7/initPipeline.py -> /group/metagen/dbtools/src/metAMOS-1.5rc3_git
changing mode of /group/metagen/dbtools/src/metAMOS-1.5rc3_git/runPipeline.py to 755
changing mode of /group/metagen/dbtools/src/metAMOS-1.5rc3_git/initPipeline.py to 755
running build_ext
skipping './src/preprocess.c' Cython extension (up-to-date)
skipping './src/annotate.c' Cython extension (up-to-date)
skipping './src/fannotate.c' Cython extension (up-to-date)
skipping './src/mapreads.c' Cython extension (up-to-date)
skipping './src/findorfs.c' Cython extension (up-to-date)
skipping './src/findscforfs.c' Cython extension (up-to-date)
skipping './src/scaffold.c' Cython extension (up-to-date)

Pushing ahead, I still try to run the test scripts, which fail because they are unable to find the test data sets, which do exist in the Test/ directory.

$ cd Test/
$ ./run_pipeline_test.sh
Warning: Celera Assembler is not found, some functionality will not be available
Warning: BLASR is not found, some functionality will not be available
Warning: SignalP+ is not found, some functionality will not be available
Warning: FRCbam is not found, some functionality will not be available
Error, provided files do not exist:
File carsonella_pe_filt.fna from library 1 does not exist

project dir /group/metagen/dbtools/src/metAMOS-1.5rc3_git/Test/test1 does not exist!
<< SNIP >>
$ ls *.fna
MA_test2.filt2.fna  carsonella_pe_filt.fna  carsonella_reference.fna

It would appear the initPipeLine script is unable to find the fna files in the Test directory. Have I missed setting a PATH?

Cheers

skoren commented 9 years ago

Hi,

Your MetaVelvet installation should be OK as it is not one of the missing tools reported when you run initPipeline. I think the required file script is reporting that erroneously.

As far as the test case, it appears your python is reporting the file as missing. Which version of python do you have? What happens if you run:

python
import os
os.path.exists("carsonella_pe_filt.fna")

from inside the Test directory? You can also try modifying run_pipeline_test.sh to use the full path to carsonella_pe_filt.fna to see if that works instead.

JustGitting commented 9 years ago

Hi,

Thanks for your reply. From within the Test directory I get the following

$ cd ./metAMOS-1.5rc3_git/Test
$ python
Python 2.7.6 (default, Mar 25 2014, 19:46:32)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.path.exists("carsonella_pe_filt.fna")
True
skoren commented 9 years ago

That is very strange because initPipeline just checks if the file is remote and if not checks os.path.exists. Can you modify initPipeline to add some debug output: line 125-126

def isSRAID(file):
   return utils.translateToSRAURL(settings, file) != ""

change to

def isSRAID(file):
   print "The path returns %s aka %s"%(utils.translateToSRAURL(settings, file), (utils.translateToSRAURL(settings, file) != ""))
   return utils.translateToSRAURL(settings, file) != ""

and line 129-130:

def isRemote(file):
   return isSRAID(file) or file.startswith("ftp://") or file.startswith("http://") or file.startswith("https://")

change to

def isRemote(file):
    print "The file %s is SRA: %s FTP: %s http %s https %s\n"%(file, isSRAID(file), file.startswith("ftp://"), file.s    tartswith("http://"),  file.startswith("https://"))
    return isSRAID(file) or file.startswith("ftp://") or file.startswith("http://") or file.startswith("https://")
JustGitting commented 9 years ago

From within the Test directory:

 $ ./run_pipeline_test.sh
Warning: Celera Assembler is not found, some functionality will not be available
Warning: BLASR is not found, some functionality will not be available
Warning: SignalP+ is not found, some functionality will not be available
Warning: FRCbam is not found, some functionality will not be available
The path returns ./carsonella_pe_filt.fna aka True
The file carsonella_pe_filt.fna is SRA: True FTP: False http False https False

The path returns ./carsonella_pe_filt.fna aka True
The path returns ./carsonella_pe_filt.fna aka True
Error, provided files do not exist:
File carsonella_pe_filt.fna from library 1 does not exist
skoren commented 9 years ago

That is indeed very strange, it seems sratools is behaving differently on your system than ours and prepending a ./ to your file name, confusing the code. I've never seen this before. Did you have sratools installed already before installing metAMOS? Do you know the version?

If you modify src/utils.py lines 1420-1421 from:

   result = getCommandOutput("%s%ssrapath %s"%(Settings.SRA, os.sep, name), True)
   if result == name:

to

   result = getCommandOutput("%s%ssrapath %s"%(Settings.SRA, os.sep, name), True)
   print "The result is %s and name is %s\n"%(result, name)
   if result == name or result == "./%s"%(name) or result == os.path.abspath(name):

Does initPipeline work?

JustGitting commented 9 years ago

Great, that worked....but the Annotation step failed.

$ ./run_pipeline_test.sh
Warning: Celera Assembler is not found, some functionality will not be available
Warning: BLASR is not found, some functionality will not be available
Warning: SignalP+ is not found, some functionality will not be available
Warning: FRCbam is not found, some functionality will not be available
The result is ./carsonella_pe_filt.fna and name is carsonella_pe_filt.fna

The result is ./carsonella_pe_filt.fna and name is carsonella_pe_filt.fna

The path returns  aka False
The result is ./carsonella_pe_filt.fna and name is carsonella_pe_filt.fna

The file carsonella_pe_filt.fna is SRA: False FTP: False http False https False

The result is ./carsonella_pe_filt.fna and name is carsonella_pe_filt.fna

The result is ./carsonella_pe_filt.fna and name is carsonella_pe_filt.fna

The path returns  aka False
The result is ./carsonella_pe_filt.fna and name is carsonella_pe_filt.fna

Project directory already exists, please specify another
Alternatively, use runPipeline to run an existing project
[Steps to be skipped]:  set(['FunctionalAnnotation', 'FindRepeats'])
Starting Task = runpipeline.RUNPIPELINE
Starting metAMOS pipeline
Warning: Celera Assembler is not found, some functionality will not be available
Warning: BLASR is not found, some functionality will not be available
Warning: SignalP+ is not found, some functionality will not be available
Warning: FRCbam is not found, some functionality will not be available
[Available RAM: 399 GB]
*ok
[Available CPUs: 20]
*ok
________________________________________
Tasks which will be run:

Task = preprocess.Preprocess
Task = assemble.SplitAssemblers
Task = assemble.Assemble
Task = assemble.CheckAsmResults
Task = assemble.SplitMappers
Task = mapreads.MapReads
Task = mapreads.CheckMapResults
Task = mapreads.SplitForORFs
Task = findorfs.FindORFS
Task = validate.Validate
Task = findreps.FindRepeats
Task = annotate.Annotate
Task = fannotate.FunctionalAnnotation
Task = scaffold.Scaffold
Task = findscforfs.FindScaffoldORFS
Task = abundance.Abundance
Task = propagate.Propagate
Task = classify.Classify
Task = postprocess.Postprocess
________________________________________
metAMOS configuration summary:
metAMOS Version:    v1.5rc3 "Praline Brownie"  workflows: core,optional,imetamos
Time and Date:      2015-02-19
Working directory:  /usr/src/metAMOS-1.5rc3_git/Test/test1
Prefix:         proba
K-Mer:          55
Threads:        15
Taxonomic level:    class
Verbose:        False
Steps to skip:      MultiAlign, FunctionalAnnotation, FindRepeats
Steps to force:     Abundance, FindORFS, Annotate, Propagate, MapReads, Assemble, FindScaffoldORFS, Classify

[citation]
MetAMOS
    Treangen, TJ ⇔  Koren, S, Sommer, DD, Liu, B, Astrovskaya, I, Ondov, B, Darling AE, Phillippy AM, Pop, M. MetAMOS: a modular and open source metagenomic assembly and analysis pipeline. Genome biology, 14(1), R2, 2013.

Step-specific configuration:
[abundance]
MetaPhyler
    /usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64
    Liu B, Gibbons T, Ghodsi M, Treangen T, Pop M. Accurate and fast estimation of taxonomic profiles from metagenomic shotgun sequences. BMC Genomics. 2011;12 Suppl 2:S4. Epub 2011 Jul 27.

[multialign]
M-GCAT
    /usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64
    Treangen TJ, Messeguer X. M-GCAT: interactively and efficiently constructing large-scale multiple genome comparison frameworks in closely related species. BMC Bioinformatics, 2006.

[fannotate]
BLAST
    /usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64
    Altschul SF, Gish W, Miller W, Myers EW, Lipman DJ. Basic local alignment search tool. J Mol Biol. 1990 Oct 5;215(3):403-10.

[scaffold]
Bambus 2
    /usr/src/metAMOS-1.5rc3_git/AMOS/Linux-x86_64/bin
    Koren S, Treangen TJ, Pop M. Bambus 2: scaffolding metagenomes. Bioinformatics 27(21): 2964-2971 2011.

[findorfs]
FragGeneScan
    /usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64
    Rho M, Tang H, Ye Y: FragGeneScan: predicting genes in short and error-prone reads. Nucleic Acids Research 2010, 38:e191-e191.

[annotate]
Kraken
    /usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64/kraken/bin
    Wood DE, Salzberg SL: Kraken: ultrafast metagenomic sequence classification using exact alignments. Genome Biology 2014, 15:R46.

[assemble]
SOAPdenovo
    /usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64
    Li Y, Hu Y, Bolund L, Wang J: State of the art de novo assembly of human genomes from massively parallel sequencing data.Human genomics 2010, 4:271-277.

[mapreads]
Bowtie
    /usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64
    Langmead B, Trapnell C, Pop M, Salzberg SL. Ultrafast and memory-efficient alignment of short DNA sequences to the human genome. Genome Biol. 2009;10(3):R25. Epub 2009 Mar 4.

[preprocess]
metAMOS built-in filtering
    N/A

[validate]
LAP
    /usr/src/metAMOS-1.5rc3_git/LAP
    Ghodsi M, Hill CM, Astrovskaya I, Lin H, Sommer DD, Koren S, Pop M. De novo likelihood-based measures for comparing genome assemblies. BMC research notes 6:334, 2013.

[other]
Krona
    /usr/src/metAMOS-1.5rc3_git/KronaTools/bin
    Ondov BD, Bergman NH, Phillippy AM. Interactive metagenomic visualization in a Web browser. BMC Bioinformatics. 2011 Sep 30;12:385.

    Job  = [[carsonella_pe_filt.fna] -> preprocess.success] completed
Completed Task = preprocess.Preprocess
    Job  = [preprocess.success -> *.run] completed
Completed Task = assemble.SplitAssemblers
    Job  = [soapdenovo.55.run -> soapdenovo.55.asm.contig] completed
Completed Task = assemble.Assemble
    Job  = [[soapdenovo.55.asm.contig] -> [assemble.ok]] completed
Completed Task = assemble.CheckAsmResults
Uptodate Task = assemble.SplitMappers
    Job  = [soapdenovo.55.asm.contig -> soapdenovo.55.contig.cvg] completed
Completed Task = mapreads.MapReads
    Job  = [[soapdenovo.55.contig.cvg] -> [mapreads.ok]] completed
Completed Task = mapreads.CheckMapResults
Uptodate Task = mapreads.SplitForORFs
    Job  = [soapdenovo.55.contig.cvg -> soapdenovo.55.faa] completed
Completed Task = findorfs.FindORFS
    Job  = [[soapdenovo.55.faa] -> [validate.ok]] completed
Completed Task = validate.Validate
    Job  = [proba.fna -> proba.repeats] completed
Completed Task = findreps.FindRepeats
Starting Task = preprocess.PREPROCESS
Starting Task = assemble.ASSEMBLE
Starting Task = mapreads.MAPREADS
Starting Task = findorfs.FINDORFS
Starting Task = validate.VALIDATE
Starting Task = findrepeats.FINDREPEATS
Starting Task = annotate.ANNOTATE
rm: cannot remove `/usr/src/metAMOS-1.5rc3_git/Test/test1/Logs/annotate.ok': No such file or directory
*****************************************************************
*************************ERROR***********************************
During annotate, the following command failed with return code 2:
>> /usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64/kraken/bin/kraken --threads 15 -db /usr/src/metAMOS-1.5rc3_git/Utilities/DB//kraken /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/in/proba.asm.contig --preload   | kraken-filter --db /usr/src/metAMOS-1.5rc3_git/Utilities/DB//kraken --threshold 0.05 > /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/out/proba.hits

*************************DETAILS***********************************
Last 10 commands run before the error (/usr/src/metAMOS-1.5rc3_git/Test/test1/Logs/COMMANDS.log)
|2015-02-19 11:10:29| touch /usr/src/metAMOS-1.5rc3_git/Test/test1/Logs/findrepeats.skip
|2015-02-19 11:10:29|# [ANNOTATE]
|2015-02-19 11:10:29| touch /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/out/proba.annots
|2015-02-19 11:10:29| unlink /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/in/proba.asm.contig
|2015-02-19 11:10:29| ln /usr/src/metAMOS-1.5rc3_git/Test/test1/Assemble/out/proba.asm.contig /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/in/proba.asm.contig
|2015-02-19 11:10:29| unlink /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/out/proba.hits
|2015-02-19 11:10:29| rm -f /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/out/*.hits
|2015-02-19 11:10:29| rm -f /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/out/*.epsilon-nb_results.txt
|2015-02-19 11:10:29| rm -f /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/out/*.phymm.out
|2015-02-19 11:10:30| /usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64/kraken/bin/kraken --threads 15 -db /usr/src/metAMOS-1.5rc3_git/Utilities/DB//kraken /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/in/proba.asm.contig --preload   | kraken-filter --db /usr/src/metAMOS-1.5rc3_git/Utilities/DB//kraken --threshold 0.05 > /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/out/proba.hits

Last 10 lines of output (/usr/src/metAMOS-1.5rc3_git/Test/test1/Logs/ANNOTATE.log)
unlink: cannot unlink `/usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/in/proba.asm.contig': No such file or directory
unlink: cannot unlink `/usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/out/proba.hits': No such file or directory
kraken-filter: can't open nodes file: No such file or directory
kraken: /usr/src/metAMOS-1.5rc3_git/Utilities/DB//kraken/database.kdb does not exist!

Please veryify input data and restart MetAMOS. If the problem persists please contact the MetAMOS development team.
*************************ERROR***********************************
*****************************************************************
Oops, MetAMOS finished with errors! see text in red above for details.
skoren commented 9 years ago

This is an error from earlier in your INSTALL.py:

Found jellyfish v2.1.3
Kraken requires jellyfish version 1

metAMOS checks for jellyfish in your path which it finds but apparently Kraken can only work with jellyfish v1 not v2. If you remove jellyfish from your path, INSTALL.py will download the correct version and build the Kraken DB. I'll update the INSTALL.py script to check jellyfish version for Kraken.

JustGitting commented 9 years ago

Sorry for late reply, but had a few other things on.

Ok.... the new INSTALL.py script had an problem finding the jellyfish version number. It was because the "jellyfish" variable, containing the jellyfish command, only contained the path to the jellyfish binary file, but not the command itself.

          jellyfish = utils.getFromPath("jellyfish", "Jellyfish", False)

          # check jellyfish version, kraken needs version 1
          version=""
          if jellyfish != "":
             version = utils.getCommandOutput("%s --version |awk '{print substr($NF, 1, index($NF, \".\")-1)}'"%(jellyfish), False)
             print "'{0}' version found: '{1}'".format(jellyfish, version)
             if int(version) > 1:
                jellyfish=""

Should be. 

          jellyfish = utils.getFromPath("jellyfish", "Jellyfish", False)

          # check jellyfish version, kraken needs version 1
          version=""
          if jellyfish != "":
             jellyfish = jellyfish + os.sep + "jellyfish"
             version = utils.getCommandOutput("%s --version |awk '{print substr($NF, 1, index($NF, \".\")-1)}'"%(jellyfish), False)
             print "'{0}' version found: '{1}'".format(jellyfish, version)
             if int(version) > 1:

Secondly, by removing the jellyfish path from PATH, resulted in python not being found... so after metAMOS installed jellyfish v1, I had to add the jellyfish bin path to the head of PATH, ie:

export PATH="/usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64/jellyfish/bin:${PATH}"

Running INSTALL.py a few times just to make sure it works, I get the following during install:

$ python INSTALL.py core imetamos optional 2>&1 | tee install.log
$ cat install.log 
running install_scripts
running egg_info
writing metAMOS.egg-info/PKG-INFO
writing top-level names to metAMOS.egg-info/top_level.txt
writing dependency_links to metAMOS.egg-info/dependency_links.txt
writing metAMOS.egg-info/PKG-INFO
writing top-level names to metAMOS.egg-info/top_level.txt
writing dependency_links to metAMOS.egg-info/dependency_links.txt
reading manifest file 'metAMOS.egg-info/SOURCES.txt'
writing manifest file 'metAMOS.egg-info/SOURCES.txt'
running build_scripts
copying build/scripts-2.7/runPipeline.py -> /usr/src/metAMOS-1.5rc3_git
copying build/scripts-2.7/initPipeline.py -> /usr/src/metAMOS-1.5rc3_git
changing mode of /usr/src/metAMOS-1.5rc3_git/runPipeline.py to 755
changing mode of /usr/src/metAMOS-1.5rc3_git/initPipeline.py to 755
running build_ext
skipping './src/preprocess.c' Cython extension (up-to-date)
skipping './src/annotate.c' Cython extension (up-to-date)
skipping './src/fannotate.c' Cython extension (up-to-date)
skipping './src/mapreads.c' Cython extension (up-to-date)
skipping './src/findorfs.c' Cython extension (up-to-date)
skipping './src/findscforfs.c' Cython extension (up-to-date)
skipping './src/scaffold.c' Cython extension (up-to-date)
<<Welcome to metAMOS install>>
Selected to install workflowName: CORE.
Selected to install workflowName: OPTIONAL.
Selected to install workflowName: IMETAMOS.
Will automatically install:
    Pysam
    Fastqc
    Ca
    Ale
    Sga
    Velvet-Sc
    Metaphyler
    Phylosift
    Metavelvet
    Velvet
    Soap2
    Frcbam
    Quast
    Eautils
    Ray
    Numpy
    Freebayes
    Fcp
    Edena
    Uniprot
    Cython
    Cgal
    Mira
    Matplotlib
    Prokka
    Idba
    Cmake
    Lap
    Masurca
    Kronatools
    Abyss
    Sra
    Spades
    Kraken
    Psutil
    Setuptools
    Amos
    Phmmer
    Kmergenie
    Reapr
Checking whether refseq_protein is complete. Expecting 17 partitions.

Run setup.py..

The order of the messages is out of order because I've used redirection and tee, but all looks ok.

So going back to the test suite:

$ cd Test
$ ./run_pipeline_test.sh 2>&1 | tee test_pipeline.log

Unfortunately it fails again for the annotation step with the Karen DB being missing. How can I install/rebuild this?

Warning: Celera Assembler is not found, some functionality will not be available
Warning: BLASR is not found, some functionality will not be available
Warning: SignalP+ is not found, some functionality will not be available
Warning: FRCbam is not found, some functionality will not be available
Project directory already exists, please specify another
Alternatively, use runPipeline to run an existing project
[Steps to be skipped]:  set(['FunctionalAnnotation', 'FindRepeats'])
Starting Task = runpipeline.RUNPIPELINE
Starting metAMOS pipeline
Warning: Celera Assembler is not found, some functionality will not be available
Warning: BLASR is not found, some functionality will not be available
Warning: SignalP+ is not found, some functionality will not be available
Warning: FRCbam is not found, some functionality will not be available
[Available RAM: 400 GB]
m   *ok
[Available CPUs: 20]
m   *ok

________________________________________
Tasks which will be run:

Task = assemble.SplitAssemblers
Task = assemble.Assemble
Task = assemble.CheckAsmResults
Task = assemble.SplitMappers
Task = mapreads.MapReads
Task = mapreads.CheckMapResults
Task = mapreads.SplitForORFs
Task = findorfs.FindORFS
Task = validate.Validate
Task = findreps.FindRepeats
Task = annotate.Annotate
Task = fannotate.FunctionalAnnotation
Task = scaffold.Scaffold
Task = findscforfs.FindScaffoldORFS
Task = abundance.Abundance
Task = propagate.Propagate
Task = classify.Classify
Task = postprocess.Postprocess
________________________________________
metAMOS configuration summary:
metAMOS Version:    v1.5rc3 "Praline Brownie"  workflows: core,optional,imetamos
Time and Date:      2015-03-10
Working directory:  /usr/src/metAMOS-1.5rc3_git/Test/test1
Prefix:         proba
K-Mer:          55
Threads:        15
Taxonomic level:    class
Verbose:        False
Steps to skip:      MultiAlign, FunctionalAnnotation, FindRepeats
Steps to force:     Abundance, FindORFS, Annotate, Propagate, MapReads, Assemble, FindScaffoldORFS, Classify

[citation]
MetAMOS
    Treangen, TJ ⇔  Koren, S, Sommer, DD, Liu, B, Astrovskaya, I, Ondov, B, Darling AE, Phillippy AM, Pop, M. MetAMOS: a modular and open source metagenomic assembly and analysis pipeline. Genome biology, 14(1), R2, 2013.

Step-specific configuration:
[abundance]
MetaPhyler
    /usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64
    Liu B, Gibbons T, Ghodsi M, Treangen T, Pop M. Accurate and fast estimation of taxonomic profiles from metagenomic shotgun sequences. BMC Genomics. 2011;12 Suppl 2:S4. Epub 2011 Jul 27.

[multialign]
M-GCAT
    /usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64
    Treangen TJ, Messeguer X. M-GCAT: interactively and efficiently constructing large-scale multiple genome comparison frameworks in closely related species. BMC Bioinformatics, 2006.

[fannotate]
BLAST
    /usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64
    Altschul SF, Gish W, Miller W, Myers EW, Lipman DJ. Basic local alignment search tool. J Mol Biol. 1990 Oct 5;215(3):403-10.

[scaffold]
Bambus 2
    /usr/src/metAMOS-1.5rc3_git/AMOS/Linux-x86_64/bin
    Koren S, Treangen TJ, Pop M. Bambus 2: scaffolding metagenomes. Bioinformatics 27(21): 2964-2971 2011.

[findorfs]
FragGeneScan
    /usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64
    Rho M, Tang H, Ye Y: FragGeneScan: predicting genes in short and error-prone reads. Nucleic Acids Research 2010, 38:e191-e191.

[annotate]
Kraken
    /usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64/kraken/bin
    Wood DE, Salzberg SL: Kraken: ultrafast metagenomic sequence classification using exact alignments. Genome Biology 2014, 15:R46.

[assemble]
SOAPdenovo
    /usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64
    Li Y, Hu Y, Bolund L, Wang J: State of the art de novo assembly of human genomes from massively parallel sequencing data.Human genomics 2010, 4:271-277.

[mapreads]
Bowtie
    /usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64
    Langmead B, Trapnell C, Pop M, Salzberg SL. Ultrafast and memory-efficient alignment of short DNA sequences to the human genome. Genome Biol. 2009;10(3):R25. Epub 2009 Mar 4.

[preprocess]
metAMOS built-in filtering
    N/A

[validate]
LAP
    /usr/src/metAMOS-1.5rc3_git/LAP
    Ghodsi M, Hill CM, Astrovskaya I, Lin H, Sommer DD, Koren S, Pop M. De novo likelihood-based measures for comparing genome assemblies. BMC research notes 6:334, 2013.

[other]
Krona
    /usr/src/metAMOS-1.5rc3_git/KronaTools/bin
    Ondov BD, Bergman NH, Phillippy AM. Interactive metagenomic visualization in a Web browser. BMC Bioinformatics. 2011 Sep 30;12:385.

    Job  = [preprocess.success -> *.run] completed
Completed Task = assemble.SplitAssemblers
    Job  = [soapdenovo.55.run -> soapdenovo.55.asm.contig] completed
Completed Task = assemble.Assemble
    Job  = [[soapdenovo.55.asm.contig] -> [assemble.ok]] completed
Completed Task = assemble.CheckAsmResults
Uptodate Task = assemble.SplitMappers
    Job  = [soapdenovo.55.asm.contig -> soapdenovo.55.contig.cvg] completed
Completed Task = mapreads.MapReads
    Job  = [[soapdenovo.55.contig.cvg] -> [mapreads.ok]] completed
Completed Task = mapreads.CheckMapResults
Uptodate Task = mapreads.SplitForORFs
    Job  = [soapdenovo.55.contig.cvg -> soapdenovo.55.faa] completed
Completed Task = findorfs.FindORFS
    Job  = [[soapdenovo.55.faa] -> [validate.ok]] completed
Completed Task = validate.Validate
    Job  = [proba.fna -> proba.repeats] completed
Completed Task = findreps.FindRepeats
Starting Task = assemble.ASSEMBLE
Starting Task = mapreads.MAPREADS
Starting Task = findorfs.FINDORFS
Starting Task = validate.VALIDATE
Starting Task = findrepeats.FINDREPEATS
Starting Task = annotate.ANNOTATE
rm: cannot remove `/usr/src/metAMOS-1.5rc3_git/Test/test1/Logs/annotate.ok': No such file or directory
*****************************************************************
*************************ERROR***********************************
During annotate, the following command failed with return code 2:
>> /usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64/kraken/bin/kraken --threads 15 -db /usr/src/metAMOS-1.5rc3_git/Utilities/DB//kraken /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/in/proba.asm.contig --preload   | kraken-filter --db /usr/src/metAMOS-1.5rc3_git/Utilities/DB//kraken --threshold 0.05 > /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/out/proba.hits

*************************DETAILS***********************************
Last 10 commands run before the error (/usr/src/metAMOS-1.5rc3_git/Test/test1/Logs/COMMANDS.log)
|2015-03-10 14:48:58| touch /usr/src/metAMOS-1.5rc3_git/Test/test1/Logs/findrepeats.skip
|2015-03-10 14:48:58|# [ANNOTATE]
|2015-03-10 14:48:58| touch /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/out/proba.annots
|2015-03-10 14:48:58| unlink /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/in/proba.asm.contig
|2015-03-10 14:48:58| ln /usr/src/metAMOS-1.5rc3_git/Test/test1/Assemble/out/proba.asm.contig /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/in/proba.asm.contig
|2015-03-10 14:48:58| unlink /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/out/proba.hits
|2015-03-10 14:48:58| rm -f /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/out/*.hits
|2015-03-10 14:48:58| rm -f /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/out/*.epsilon-nb_results.txt
|2015-03-10 14:48:58| rm -f /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/out/*.phymm.out
|2015-03-10 14:48:58| /usr/src/metAMOS-1.5rc3_git/Utilities/cpp/Linux-x86_64/kraken/bin/kraken --threads 15 -db /usr/src/metAMOS-1.5rc3_git/Utilities/DB//kraken /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/in/proba.asm.contig --preload   | kraken-filter --db /usr/src/metAMOS-1.5rc3_git/Utilities/DB//kraken --threshold 0.05 > /usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/out/proba.hits

Last 10 lines of output (/usr/src/metAMOS-1.5rc3_git/Test/test1/Logs/ANNOTATE.log)
unlink: cannot unlink `/usr/src/metAMOS-1.5rc3_git/Test/test1/Annotate/out/proba.hits': No such file or directory
kraken-filter: can't open nodes file: No such file or directory
kraken: /usr/src/metAMOS-1.5rc3_git/Utilities/DB//kraken/database.kdb does not exist!

Please veryify input data and restart MetAMOS. If the problem persists please contact the MetAMOS development team.
*************************ERROR***********************************
*****************************************************************
Oops, MetAMOS finished with errors! see text in red above for details.
skoren commented 9 years ago

Sorry for the late reply, if you're still having this issue, I'd suggest removing kraken folders under Utilities (there should be one in Utilities/cpp/Linux-amd64/kraken and Utilities/DB/kraken and re-running INSTALL.py which should re-install kraken. Make sure jellyfish v2 is not in your path so that metAMOS install/downloads v1 of jellyfish. I've also committed a fix to check jellyfish version when building Kraken. I am closing the issue, please reopen if you continue to experience errors.