search

marbl / metAMOS

A metagenomic and isolate assembly and analysis pipeline built with AMOS
http://marbl.github.io/metAMOS
Other
93 stars 45 forks source link

During propagate MetAMOS failed with return code 134 #242

Open KseniaErshova opened 8 years ago

KseniaErshova commented 8 years ago

Hi! I tried to install MetAMOS on cluster. I just run installation commands: $ wget https://github.com/marbl/metAMOS/archive/v1.5rc3.zip $ unzip v1.5rc3.zip cd metAMOS-v1.5rc3 (BTW, here is a mistake in installation guide http://metamos.readthedocs.io/en/v1.5rc3/content/installation.html, should be "cd metAMOS-1.5rc3") $ python INSTALL.py core $ python INSTALL.py iMetAMOS $ python INSTALL.py optional

Then, during running test ($ ./run_pipeline_test.sh) I got the following error:

During propagate, the following command failed with return code 134:

/home/Ksenia.Ershova/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin/FilterEdgesByCluster -b /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Scaffold/in/proba.bnk -clusters /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Propagate/in/proba.clusters -noRemoveEdges > /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Propagate/out/proba.clusters

****DETAILS****** Last 10 commands run before the error (/home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Logs/COMMANDS.log) |2016-06-04 09:48:41|# [ABUNDANCE] |2016-06-04 09:48:41| ln -s /home/Ksenia.Ershova/metAMOS-1.5rc3/Utilities/DB//markers.pfasta /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Abundance/out/markers.pfasta |2016-06-04 09:48:42| /home/Ksenia.Ershova/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/formatdb -p T -i /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Abundance/out/markers.pfasta |2016-06-04 09:48:44| /home/Ksenia.Ershova/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/blastall -p blastp -i /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/FindORFS/out/proba.faa -d /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Abundance/out/markers.pfasta -m 8 -b 10 -v 10 -a 15 -o /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Abundance/out/proba.blastp |2016-06-04 09:48:45| perl /home/Ksenia.Ershova/metAMOS-1.5rc3/Utilities/perl/metaphyler_contigs.pl /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Abundance/out/proba.blastp proba /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/FindORFS/out/proba.gene.cvg /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Abundance/out /home/Ksenia.Ershova/metAMOS-1.5rc3/Utilities |2016-06-04 09:48:52| perl /home/Ksenia.Ershova/metAMOS-1.5rc3/Utilities/perl/ImportMetaPhyler.pl -c -v -i -p /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Abundance/out/proba.taxprof.pct.txt:class |2016-06-04 09:48:53|# [PROPAGATE] |2016-06-04 09:48:53| ln /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Annotate/out/proba.annots /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Propagate/in/proba.annots |2016-06-04 09:48:53| cat /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Propagate/in/proba.annots | grep -v "\"" | grep -v contigID |sed s/utg//g |sed s/ctg//g > /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Propagate/in/proba.clusters |2016-06-04 09:48:53| /home/Ksenia.Ershova/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin/FilterEdgesByCluster -b /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Scaffold/in/proba.bnk -clusters /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Propagate/in/proba.clusters -noRemoveEdges > /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Propagate/out/proba.clusters

Last 10 lines of output (/home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Logs/PROPAGATE.log) ln: создание жесткой ссылки «/home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Propagate/in/proba.annots»: Файл существует Options summary: Bank = /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Scaffold/in/proba.bnk WARNING: link 7 is marked as bad in the bank, skipping WARNING: link 9 is marked as bad in the bank, skipping Iteration 0 terminate called after throwing an instance of 'AMOS::IOException_t' what(): Unknown file write error in concat, bank corrupted /bin/bash: line 1: 11241 Аварийный останов (core dumped) /home/Ksenia.Ershova/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin/FilterEdgesByCluster -b /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Scaffold/in/proba.bnk -clusters /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Propagate/in/proba.clusters -noRemoveEdges > /home/Ksenia.Ershova/metAMOS-1.5rc3/Test/test1/Propagate/out/proba.clusters

Please veryify input data and restart MetAMOS. If the problem persists please contact the MetAMOS development team. ****ERROR******

There were no problems with CPU and RAM [Available RAM: 198 GB] ok [Available CPUs: 16] ok

But it still doesn't work.

Please, help me to manage this problem. I really need to run MetAMOS asap.

Thank you in advance.

KseniaErshova commented 8 years ago

Second attempt to install metAMOS gave me this: curl: (7) couldn't connect to host tar (child): stat.tar.gz: Функция open завершилась с ошибкой: Нет такого файла или каталога tar (child): Error is not recoverable: exiting now tar: Child returned status 2 tar: Error is not recoverable: exiting now Traceback (most recent call last): File "INSTALL.py", line 475, in os.chdir("Statistics-Descriptive-3.0203") OSError: [Errno 2] No such file or directory: 'Statistics-Descriptive-3.0203'

Could you please help to install it properly?

skoren commented 8 years ago

If you change the download command from: os.system("curl -L ftp://cbcb.umd.edu/pub/data/metamos/Statistics-Descriptive-3.0203.tar.gz -o stat.tar.gz")

to os.system("curl -L ftp://ftp.cbcb.umd.edu/pub/data/metamos/Statistics-Descriptive-3.0203.tar.gz -o stat.tar.gz")

It should work

KseniaErshova commented 8 years ago

Yea, it works for INSTALL.py core option. Thank you. But when I tried to run python INSTALL.py iMetAMOS I got new problem:

Appending installation info to /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/File-Copy-Link-0.113/build/lib64/perl5/perllocal.pod cp: невозможно выполнить stat для «install.sh»: Нет такого файла или каталога Traceback (most recent call last): File "INSTALL.py", line 1631, in testIn = open("install.sh.orig", 'r') IOError: [Errno 2] No such file or directory: 'install.sh.orig'

What should I change next?

skoren commented 8 years ago

It's caused by the same thing, the FTP server hosting the files invalided some paths in the release. Whenever you see ftp://cbcb.umd.edu change it to ftp://ftp.cbcb.umd.edu/

KseniaErshova commented 8 years ago

Thank you! I changed each "ftp://cbcb.umd.edu" in INSTALL.py file. It helped to install core and iMetAMOS. But the problem with self-testing still persists: During propagate, the following command failed with return code 134:

/home/Ksenia.Ershova/soft/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin/FilterEdgesByCluster -b /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Scaffold/in/proba.bnk -clusters /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Propagate/in/proba.clusters -noRemoveEdges > /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Propagate/out/proba.clusters

****DETAILS****** Last 10 commands run before the error (/home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Logs/COMMANDS.log) |2016-06-08 07:40:58|# [ABUNDANCE] |2016-06-08 07:40:58| ln -s /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Utilities/DB//markers.pfasta /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Abundance/out/markers.pfasta |2016-06-08 07:40:59| /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/formatdb -p T -i /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Abundance/out/markers.pfasta |2016-06-08 07:41:01| /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Utilities/cpp/Linux-x86_64/blastall -p blastp -i /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/FindORFS/out/proba.faa -d /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Abundance/out/markers.pfasta -m 8 -b 10 -v 10 -a 15 -o /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Abundance/out/proba.blastp |2016-06-08 07:41:02| perl /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Utilities/perl/metaphyler_contigs.pl /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Abundance/out/proba.blastp proba /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/FindORFS/out/proba.gene.cvg /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Abundance/out /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Utilities |2016-06-08 07:41:10| perl /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Utilities/perl/ImportMetaPhyler.pl -c -v -i -p /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Abundance/out/proba.taxprof.pct.txt:class |2016-06-08 07:41:11|# [PROPAGATE] |2016-06-08 07:41:11| ln /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Annotate/out/proba.annots /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Propagate/in/proba.annots |2016-06-08 07:41:11| cat /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Propagate/in/proba.annots | grep -v "\"" | grep -v contigID |sed s/utg//g |sed s/ctg//g > /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Propagate/in/proba.clusters |2016-06-08 07:41:11| /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin/FilterEdgesByCluster -b /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Scaffold/in/proba.bnk -clusters /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Propagate/in/proba.clusters -noRemoveEdges > /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Propagate/out/proba.clusters

Last 10 lines of output (/home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Logs/PROPAGATE.log) Options summary: Bank = /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Scaffold/in/proba.bnk WARNING: link 7 is marked as bad in the bank, skipping WARNING: link 9 is marked as bad in the bank, skipping Iteration 0 terminate called after throwing an instance of 'AMOS::IOException_t' what(): Unknown file write error in concat, bank corrupted /bin/bash: line 1: 7897 Аварийный останов (core dumped) /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/AMOS/Linux-x86_64/bin/FilterEdgesByCluster -b /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Scaffold/in/proba.bnk -clusters /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Propagate/in/proba.clusters -noRemoveEdges > /home/Ksenia.Ershova/soft/metAMOS-1.5rc3/Test/test1/Propagate/out/proba.clusters

Please veryify input data and restart MetAMOS. If the problem persists please contact the MetAMOS development team. ****ERROR******

skoren commented 8 years ago

This looks like a file system access error, make sure you're not out of space on the disk.