search

marbl / metAMOS

A metagenomic and isolate assembly and analysis pipeline built with AMOS
http://marbl.github.io/metAMOS
Other
93 stars 45 forks source link

Preprocess error #174

Closed Biomicrogen closed 9 years ago

Biomicrogen commented 9 years ago

Hello,

I tried running the test scripts and I couldn't get passed the preprocess step, I am guessing I am missing one of the processing dependancies. I installed the core and the iMetAMos from the install.pr file.

Here is the error: During preprocess, the following command failed with return code 127:

java -cp /home/biomicrogen/metamos/Utilities/java:. convertFastqToFasta /home/biomicrogen/metamos/Test/test12/Preprocess/out/lib1.seq /home/biomicrogen/metamos/Test/test12/Preprocess/out/lib1.fasta /home/biomicrogen/metamos/Test/test12/Preprocess/out/lib1.fasta.qual

****DETAILS****** Last 10 commands run before the error (/home/biomicrogen/metamos/Test/test12/Logs/COMMANDS.log) |2014-12-30 16:11:25| touch /home/biomicrogen/metamos/Test/test12/Preprocess/in/test2.fq |2014-12-30 16:11:26|# [PREPROCESS] |2014-12-30 16:11:26| rm /home/biomicrogen/metamos/Test/test12/Preprocess/out/all.seq.mates |2014-12-30 16:11:27| perl /home/biomicrogen/metamos/Utilities/perl/shuffleSequences_fastq.pl /home/biomicrogen/metamos/Test/test12/Preprocess/out/test1.fq /home/biomicrogen/metamos/Test/test12/Preprocess/out/test2.fq /home/biomicrogen/metamos/Test/test12/Preprocess/out/lib1.seq |2014-12-30 16:11:27| ln /home/biomicrogen/metamos/Test/test12/Preprocess/out/test1.fq /home/biomicrogen/metamos/Test/test12/Preprocess/out/lib1.1.fastq |2014-12-30 16:11:27| ln /home/biomicrogen/metamos/Test/test12/Preprocess/out/test2.fq /home/biomicrogen/metamos/Test/test12/Preprocess/out/lib1.2.fastq |2014-12-30 16:11:27| which python |2014-12-30 16:11:27| echo $PYTHONPATH |2014-12-30 16:11:27| ln /home/biomicrogen/metamos/Test/test12/Preprocess/out/lib1.seq /home/biomicrogen/metamos/Test/test12/Preprocess/out/lib1.fastq |2014-12-30 16:11:27| java -cp /home/biomicrogen/metamos/Utilities/java:. convertFastqToFasta /home/biomicrogen/metamos/Test/test12/Preprocess/out/lib1.seq /home/biomicrogen/metamos/Test/test12/Preprocess/out/lib1.fasta /home/biomicrogen/metamos/Test/test12/Preprocess/out/lib1.fasta.qual

Last 10 lines of output (/home/biomicrogen/metamos/Test/test12/Logs/PREPROCESS.log) rm: cannot remove ‘/home/biomicrogen/metamos/Test/test12/Preprocess/out/all.seq.mates’: No such file or directory /usr/bin/python /home/biomicrogen/metamos/Utilities/python:/home/biomicrogen/metamos/Utilities/ruffus:/home/biomicrogen/metamos/Utilities/python/lib:/home/biomicrogen/metamos/Utilities/python/lib/python:/home/biomicrogen/metamos/Utilities/python/lib64:/home/biomicrogen/metamos/Utilities/python/lib64/python:/home/biomicrogen/metamos/Utilities: /bin/bash: java: command not found

Please veryify input data and restart MetAMOS. If the problem persists please contact the MetAMOS development team. ****ERROR******

Any help would be fantastic!

Cheers, Biomicrogen

treangen commented 9 years ago

Hello,

The error indicates you do not have Java installed on your system, however, Java is a prerequisite for running MetAMOS, so you will need to download & install Java first.

This was not clearly indicated in the documentation, I've just updated the readthedocs page to now highlight this dependency: http://metamos.readthedocs.org/en/v1.5rc3/content/installation.html#prerequisites

Biomicrogen commented 9 years ago

That is what I thought so, that is fixed now I think I am getting a KronaTools error. I installed the core plus Kraken and KronaTools and still got the error. see below. I really appreciate your fast response!

During annotate, the following command failed with return code 2:

perl /home/biomicrogen/metamos/Utilities/perl/ImportKraken.pl -c -i -f /home/biomicrogen/metamos/Test/test1/Annotate/in/proba.asm.contig /home/biomicrogen/metamos/Test/test1/Annotate/out/proba.hits:/home/biomicrogen/metamos/Test/test1/Assemble/out/proba.contig.cnt:class

****DETAILS****** Last 10 commands run before the error (/home/biomicrogen/metamos/Test/test1/Logs/COMMANDS.log) |2014-12-30 17:07:29|# [ANNOTATE] |2014-12-30 17:07:29| touch /home/biomicrogen/metamos/Test/test1/Annotate/out/proba.annots |2014-12-30 17:07:29| unlink /home/biomicrogen/metamos/Test/test1/Annotate/in/proba.asm.contig |2014-12-30 17:07:29| ln /home/biomicrogen/metamos/Test/test1/Assemble/out/proba.asm.contig /home/biomicrogen/metamos/Test/test1/Annotate/in/proba.asm.contig |2014-12-30 17:07:29| unlink /home/biomicrogen/metamos/Test/test1/Annotate/out/proba.hits |2014-12-30 17:07:29| rm -f /home/biomicrogen/metamos/Test/test1/Annotate/out/.hits |2014-12-30 17:07:29| rm -f /home/biomicrogen/metamos/Test/test1/Annotate/out/.epsilon-nb_results.txt |2014-12-30 17:07:29| rm -f /home/biomicrogen/metamos/Test/test1/Annotate/out/*.phymm.out |2014-12-30 17:07:33| /home/biomicrogen/metamos/Utilities/cpp/Linux-x86_64/kraken/bin/kraken --threads 15 -db /home/biomicrogen/metamos/Utilities/DB//kraken /home/biomicrogen/metamos/Test/test1/Annotate/in/proba.asm.contig --preload | kraken-filter --db /home/biomicrogen/metamos/Utilities/DB//kraken --threshold 0.05 > /home/biomicrogen/metamos/Test/test1/Annotate/out/proba.hits |2014-12-30 17:07:33| perl /home/biomicrogen/metamos/Utilities/perl/ImportKraken.pl -c -i -f /home/biomicrogen/metamos/Test/test1/Annotate/in/proba.asm.contig /home/biomicrogen/metamos/Test/test1/Annotate/out/proba.hits:/home/biomicrogen/metamos/Test/test1/Assemble/out/proba.contig.cnt:class

Last 10 lines of output (/home/biomicrogen/metamos/Test/test1/Logs/ANNOTATE.log) unlink: cannot unlink ‘/home/biomicrogen/metamos/Test/test1/Annotate/out/proba.hits’: No such file or directory Loading database... complete. 3 sequences (0.16 Mbp) processed in 2.785s (0.1 Kseq/m, 3.44 Mbp/m). 3 sequences classified (100.00%) 0 sequences unclassified (0.00%) Can't locate KronaTools.pm in @INC (you may need to install the KronaTools module) (@INC contains: /home/biomicrogen/metamos/src/phylosift/lib/ /etc/perl /usr/local/lib/perl/5.18.2 /usr/local/share/perl/5.18.2 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.18 /usr/share/perl/5.18 /usr/local/lib/site_perl .) at /home/biomicrogen/metamos/Utilities/perl/ImportKraken.pl line 14. BEGIN failed--compilation aborted at /home/biomicrogen/metamos/Utilities/perl/ImportKraken.pl line 14.

Please veryify input data and restart MetAMOS. If the problem persists please contact the MetAMOS development team.

skoren commented 9 years ago

This looks like KronaTools did not install successfully when you ran INSTALL.py. Can you confirm you have a KronaTools folder in your metAMOS subdirectory and that it has a lib folder and bin folder:

bash-3.2$ ls -lha KronaTools/lib/
total 80
drwxr-xr-x+  3 KorenS  staff   102B Nov 24 18:07 .
drwxr-xr-x+ 14 KorenS  staff   476B Nov 24 18:07 ..
-rw-r--r--@  1 KorenS  staff    36K Aug 12  2013 KronaTools.pm

bash-3.2$ ls KronaTools/bin/
ktClassifyBLAST     ktImportBLAST       ktImportGalaxy      ktImportPhymmBL     ktImportText
ktGetContigMagnitudes   ktImportDiskUsage   ktImportMETAREP-BLAST   ktImportRDP     ktImportXML
ktGetLibPath        ktImportEC      ktImportMETAREP-EC  ktImportRDPComparison
ktGetTaxIDFromGI    ktImportFCP     ktImportMGRAST      ktImportTaxonomy

If it is missing, you can try re-installing it. First, make sure the KronaTools folder is removed from metAMOS and re-run INSTALL.py and make sure your INSTALL.py references KronaTools 2.4 on line 124, not 2.2. You can also try installing the latest KronaTools manually. Please report any errors that the INSTALL.py command reports and the exact command line you used to run INSTALL.py.

Biomicrogen commented 9 years ago

Ok, the KronaTools folder was there and I reinstalled and still got the error. I uninstalled everything and removed the metamos directory completely. I will rebuild everything. I want to install the entire package with AllDBs. I've already dl the allDB tar file. The instructions for installing the complete MetAmos distro are a bit fuzzy. Should I run 'python INSTALL.py optional' to install all workflows? Were do I incorporate the ALLDB.tar file contents?

Thanks,

Brandon S. Guida Mc.S., M.S. PADI - Divemaster President - ASU Devil Divers Arizona State University Ph.D Candidate - Garcia-Pichel Geomicrobiology Lab

Mesa Community College - Professor Microbiology, A&P, and Biotechnology

"It is not enough to just believe what you see, you must understand what you see." - Leonardo Da Vinci

"A human being should be able to change a diaper, plan an invasion, butcher a hog, conn a ship, design a building, write a sonnet, balance accounts, build a wall, set a bone, comfort the dying, take orders, give orders, cooperate, act alone, solve equations, analyze a new problem, pitch manure, program a computer, cook a tasty meal, fight efficiently, die gallantly. Specialization is for insects." - Robert Heinlein.

-----Original Message----- From: Sergey Koren notifications@github.com To: marbl/metAMOS metAMOS@noreply.github.com Cc: Biomicrogen biomicrogen@aol.com Sent: Mon, Jan 5, 2015 7:21 am Subject: Re: [metAMOS] Preprocess error (#174)

This looks like KronaTools did not install successfully when you ran INSTALL.py. Can you confirm you have a KronaTools folder in your metAMOS subdirectory and that it has a lib folder:

bash-3.2$ ls -lha KronaTools/lib/ total 80 drwxr-xr-x+ 3 KorenS staff 102B Nov 24 18:07 . drwxr-xr-x+ 14 KorenS staff 476B Nov 24 18:07 .. -rw-r--r--@ 1 KorenS staff 36K Aug 12 2013 KronaTools.pm

If it is missing, you can try re-installing it. First, make sure the KronaTools folder is removed from metAMOS and re-run INSTALL.py. You can also try installing the latest KronaTools manually. Please report any errors that the INSTALL.py command reports and the exact command line you used to run INSTALL.py. — Reply to this email directly or view it on GitHub.

skoren commented 9 years ago

Hi,

The ALLDB file is meant to be used with the frozen binary (http://metamos.readthedocs.org/en/v1.5rc3/content/frozenbinary.html). If you are using the INSTALL.py method, the databases will be downloaded/installed by the script. If you would like to install everything available with metAMOS (including the assemblers/validators which are part of iMetAMOS) run:

python INSTALL.py core optional imetamos

If you do not want the assemblers which come with iMetAMOS run

python INSTALL.py core optional
Biomicrogen commented 9 years ago

How is the ALLDB file used with the binary? It doesn't specify usage. Do I have to put the ALLDB file in the metamos parent folder before I install? I think I am missing something, sorry. I appreciate your help, a lot!

Brandon

-----Original Message----- From: Sergey Koren notifications@github.com To: marbl/metAMOS metAMOS@noreply.github.com Cc: Biomicrogen biomicrogen@aol.com Sent: Tue, Jan 6, 2015 11:55 am Subject: Re: [metAMOS] Preprocess error (#174)

Hi, The ALLDB file is meant to be used with the frozen binary (http://metamos.readthedocs.org/en/v1.5rc3/content/frozenbinary.html). If you are using the INSTALL.py method, the databases will be downloaded/installed by the script. If you would like to install everything available with metAMOS (including the assemblers/validators which are part of iMetAMOS) run: python INSTALL.py core optional imetamos

If you do not want the assemblers which come with iMetAMOS run python INSTALL.py core optional

— Reply to this email directly or view it on GitHub.

skoren commented 9 years ago

Sorry for not replying earlier, I assume you have been able to install the DB directory. In case other users encounter a similar issue, the ALLDB file should be extracted to the same directory as your runPipeline/initPipeline files. It will create a folder named DB in that subdirectory which will automatically be found by the frozen binary.