search

marbl / metAMOS

A metagenomic and isolate assembly and analysis pipeline built with AMOS
http://marbl.github.io/metAMOS
Other
93 stars 45 forks source link

error at the postprocess step #240

Closed duanjunhyq closed 8 years ago

duanjunhyq commented 8 years ago

Hi, I installed metAMOS-1.5rc3 in our sever by using "python INSTALL.py core". Then I tried to run the test script of run_pipeline_test.sh. However, it was stuck at the postprocess step without any hit for errors. Could you help to solve this issue? Thank you.

Best, Jun

Task = postprocess.POSTPROCESS Oops, MetAMOS finished with errors! see text in red above for details.

[root@localhost Test]# more log.txt Warning: Celera Assembler is not found, some functionality will not be available Warning: BLASR is not found, some functionality will not be available Warning: Newbler is not found, some functionality will not be available Warning: MetaGeneMark is not found, some functionality will not be available Warning: metaphylerClassify is not found, some functionality will not be available Warning: PhyloSift was not found, will not be available

Warning: EA-UTILS is not found, some functionality will not be available Warning: KmerGenie is not found, some functionality will not be available Warning: ALE is not found, some functionality will not be available Warning: CGAL is not found, some functionality will not be available Warning: REAPR is not found, some functionality will not be available Warning: FRCbam is not found, some functionality will not be available Warning: FreeBayes is not found, some functionality will not be available Warning: QUAST is not found, some functionality will not be available Project directory already exists, please specify another Alternatively, use runPipeline to run an existing project

Starting Task = runpipeline.RUNPIPELINE Starting metAMOS pipeline Error: cannot find BLAST DB directory, expected it in /home/will/metAMOS-1.5rc3/Utilities/DB/. Disabling blastdb dependent programs Warning: Celera Assembler is not found, some functionality will not be available Warning: BLASR is not found, some functionality will not be available Warning: Newbler is not found, some functionality will not be available Warning: MetaGeneMark is not found, some functionality will not be available Warning: metaphylerClassify is not found, some functionality will not be available Warning: PhyloSift was not found, will not be available

Warning: EA-UTILS is not found, some functionality will not be available Warning: KmerGenie is not found, some functionality will not be available Warning: ALE is not found, some functionality will not be available Warning: CGAL is not found, some functionality will not be available Warning: REAPR is not found, some functionality will not be available Warning: FRCbam is not found, some functionality will not be available Warning: FreeBayes is not found, some functionality will not be available Warning: QUAST is not found, some functionality will not be available [Available RAM: 96 GB] ok [Available CPUs: 12] ok

Tasks which will be run:

Task = assemble.SplitAssemblers Task = assemble.Assemble Task = assemble.CheckAsmResults Task = assemble.SplitMappers Task = mapreads.MapReads Task = mapreads.CheckMapResults Task = mapreads.SplitForORFs Task = findorfs.FindORFS Task = validate.Validate Task = findreps.FindRepeats Task = annotate.Annotate Task = fannotate.FunctionalAnnotation Task = scaffold.Scaffold Task = findscforfs.FindScaffoldORFS Task = abundance.Abundance Task = propagate.Propagate Task = classify.Classify Task = postprocess.Postprocess

metAMOS configuration summary: metAMOS Version: v1.5rc3 "Praline Brownie" workflows: core Time and Date: 2016-04-21 Working directory: /home/will/metAMOS-1.5rc3/Test/test1 Prefix: proba K-Mer: 55 Threads: 15 Taxonomic level: class Verbose: False Steps to skip: MultiAlign, FunctionalAnnotation, FindRepeats Steps to force: Abundance, FindORFS, Annotate, Propagate, MapReads, Assemble, FindScaffoldORFS, Classify

[citation] MetAMOS Treangen, TJ ? Koren, S, Sommer, DD, Liu, B, Astrovskaya, I, Ondov, B, Darling AE, Phillippy AM, Pop, M. MetAMOS: a modular and open source metagenomic assembly and analysis pipeline. Genome biology, 14(1), R2, 2013.

Step-specific configuration: [abundance] MetaPhyler

    Liu B, Gibbons T, Ghodsi M, Treangen T, Pop M. Accurate and fast estimation of taxonomic profiles from metagenomic shotgun s

equences. BMC Genomics. 2011;12 Suppl 2:S4. Epub 2011 Jul 27.

[multialign] M-GCAT /home/will/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64 Treangen TJ, Messeguer X. M-GCAT: interactively and efficiently constructing large-scale multiple genome comparison framewor ks in closely related species. BMC Bioinformatics, 2006.

[fannotate] BLAST /home/will/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64 Altschul SF, Gish W, Miller W, Myers EW, Lipman DJ. Basic local alignment search tool. J Mol Biol. 1990 Oct 5;215(3):403-10.

[scaffold] Bambus 2 /home/will/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin Koren S, Treangen TJ, Pop M. Bambus 2: scaffolding metagenomes. Bioinformatics 27(21): 2964-2971 2011.

[findorfs] FragGeneScan /home/will/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64 Rho M, Tang H, Ye Y: FragGeneScan: predicting genes in short and error-prone reads. Nucleic Acids Research 2010, 38:e191-e19 1.

[annotate] Kraken /home/will/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/kraken/bin Wood DE, Salzberg SL: Kraken: ultrafast metagenomic sequence classification using exact alignments. Genome Biology 2014, 15: R46.

[assemble] SOAPdenovo /home/will/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64 Li Y, Hu Y, Bolund L, Wang J: State of the art de novo assembly of human genomes from massively parallel sequencing data.Hum an genomics 2010, 4:271-277.

[mapreads] Bowtie /home/will/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64 Langmead B, Trapnell C, Pop M, Salzberg SL. Ultrafast and memory-efficient alignment of short DNA sequences to the human gen ome. Genome Biol. 2009;10(3):R25. Epub 2009 Mar 4.

[preprocess] metAMOS built-in filtering N/A

[validate] LAP /home/will/metAMOS-1.5rc3/LAP Ghodsi M, Hill CM, Astrovskaya I, Lin H, Sommer DD, Koren S, Pop M. De novo likelihood-based measures for comparing genome a ssemblies. BMC research notes 6:334, 2013.

[other] Krona /home/will/metAMOS-1.5rc3/KronaTools/bin Ondov BD, Bergman NH, Phillippy AM. Interactive metagenomic visualization in a Web browser. BMC Bioinformatics. 2011 Sep 30; 12:385.

Starting Task = assemble.ASSEMBLE Starting Task = mapreads.MAPREADS Starting Task = findorfs.FINDORFS Starting Task = validate.VALIDATE Starting Task = findrepeats.FINDREPEATS Starting Task = annotate.ANNOTATE Starting Task = functionalannotation.FUNCTIONALANNOTATION Starting Task = scaffold.SCAFFOLD Starting Task = findscaffoldorfs.FINDSCAFFOLDORFS Starting Task = abundance.ABUNDANCE Starting Task = propagate.PROPAGATE Starting Task = classify.CLASSIFY Starting Task = postprocess.POSTPROCESS Oops, MetAMOS finished with errors! see text in red above for details.

skoren commented 8 years ago

This is most likely a missing perl library, add the -v option to runPipeline in the script and it will give more output as it runs.

treangen commented 8 years ago

hi Jun,

you could also view the POSTPROCESS.log file within the ./Log directory, which may help to identify the issue, in addition to re-running with '-v' to runPipeline.

duanjunhyq commented 8 years ago

Hi Sergey and Todd,

Thank you for your quick response. I have checked the log file but it provided little hints. The log file just said it couldn't unlink some files. I think it's normal because some files couldn't be produced as the pipeline was stuck at the intermediate step，so the program couldn't find these files to remove.

When I add -v to runPipeline, it provided the following information. I'm still not sure how to solve it.

Thanks.

Jun

###########################################

Oops, MetAMOS finished with errors! see text in red above for details. Traceback (most recent call last): File "../runPipeline", line 985, in verbose = 1) File "/home/will/metAMOS-1.5rc3/Utilities/ruffus/task.py", line 2965, in pipeline_run raise job_errors RethrownJobError: Exception #1 'exceptions.TypeError(histogram() got an unexpected keyword argument 'new')' raised in ... Task = def postprocess.Postprocess(...): Job = [proba.asm.contig -> proba.scf.fa]

Traceback (most recent call last):
  File "/home/will/metAMOS-1.5rc3/Utilities/ruffus/task.py", line 625, in run_pooled_job_without_exceptions
    return_value =  job_wrapper(param, user_defined_work_func, register_cleanup, touch_files_only)
  File "/home/will/metAMOS-1.5rc3/Utilities/ruffus/task.py", line 491, in job_wrapper_io_files
    ret_val = user_defined_work_func(*param)
  File "/home/will/metAMOS-1.5rc3/src/postprocess.py", line 286, in Postprocess
    create_summary("%s/Abundance/out/%s.taxprof.pct.txt"%(_settings.rundir,_settings.PREFIX),"%s/Postprocess/out/%s.bnk"%(_settings.rundir,_settings.PREFIX),"%s/Postprocess/out/html/"%(_settings.rundir),"%s/Postprocess/out/%s.scf.fa"%(_settings.rundir,_settings.PREFIX),"%s"%(_settings.METAMOS_UTILS),"%s/img"%(_settings.METAMOSDIR),"%s"%(_settings.AMOS),len(_readlibs),"%s"%(_settings.taxa_level),"%s"%(_settings.DB_DIR))
  File "/home/will/metAMOS-1.5rc3/Utilities/python/create_summary.py", line 234, in create_summary
    create_plots("%s/plot.tab"%(html_prefix),"%s"%("proba1"))
  File "/home/will/metAMOS-1.5rc3/Utilities/python/create_plots.py", line 165, in create_plots
    n, bins, patches = plt.hist(h,  100)
  File "/usr/lib64/python2.7/site-packages/matplotlib/pyplot.py", line 2657, in hist
    stacked=stacked, **kwargs)
  File "/usr/lib64/python2.7/site-packages/matplotlib/axes.py", line 8101, in hist
    m, bins = np.histogram(x[i], bins, weights=w[i], **hist_kwargs)
TypeError: histogram() got an unexpected keyword argument 'new'
skoren commented 8 years ago

This looks like a matplotlib version issue, metAMOS has only been tested with version 1.3 (see documentation here: http://metamos.readthedocs.io/en/latest/content/installation.html), newer version may not work so I'd suggest installing an older version to see if it fixes the error.

skoren commented 8 years ago

Closed, inactivity