search

fanhuan / AAF

alignment and assembly free
13 stars 4 forks source link

AAF nonparametric bootstrap issue #1

Open dabitz opened 7 years ago

dabitz commented 7 years ago

Hi, I am having some trouble to run the nonparametric bootstrap on my data. Could you help me? Below you find the error message.

rgufal@cln0:~/Documentos/ANDRE/AAF$ python2 BetaVersion/nonparametric_bootstrap.py -k 25 -t 4 -G 16 -d data --S1 100 --S2 20 SPECIES LIST: Pangu Pgaud Pgram Psp1H Psp2H Pstri Pten1 Pten2 Purvi Raber Ralb1 Ralb2 Rbald Rbar1 Rbar2 Rbert Rbifl Rbrac Rbras Rbro1 Rbro2 Rbro3 Rbuch Rcali Rcap1 Rcap2 Rceph Rceps Rchal Rchap Rchin Rcili Rcilo Rcolo Rcoma Rcomp Rcrin Rcube Rdebi Rdecu Rdist Rdiva Rdive Relat Relli Remac Rexal Rexim Rfasc Rfern Rfili Rfusc Rgale Rglaz Rglo1 Rglo2 Rglo3 Rgray Rharp Rharv Rimer Rinex Rinte Rknie Rlati Rlava Rlept Rmacr Rmari Rmarl Rmega Rmegp Rmicp Rmicr Rmili Rmixt Rnite Rnive Rodor Rpall Rperp Rplum Rpoly Rprui Rpube Rpunc Rpusi Rrace Rradi Rrari Rreco Rried Rripa Rrobu Rrubr Rschi Rscir Rsesl Rsier RspI1 Rsten Rsulc Rtene Rtenu Rtrac Rvulc Rwrig 1 out of 100 times of bootstrap over reads.

Thu Feb 2 00:16:27 2017 running batch 1/27 kmer_count -l 25 -n 1 -G 4 -o data/Pangu.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Pangu/Pangu.fa' kmer_count -l 25 -n 1 -G 4 -o data/Pgaud.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Pgaud/Pgaud.fa' kmer_count -l 25 -n 1 -G 4 -o data/Pgram.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Pgram/Pgram.fa' kmer_count -l 25 -n 1 -G 4 -o data/Psp1H.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Psp1H/Psp1H.fa' Total: 236590 kmers Total: 273945 kmers Total: 405446 kmers Total: 436814 kmers

kmer_count -l 25 -n 1 -G 4 -o data/Psp2H.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Psp2H/Psp2H.fa' kmer_count -l 25 -n 1 -G 4 -o data/Pstri.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Pstri/Pstri.fa' Thu Feb 2 00:16:28 2017 running batch 2/27 kmer_count -l 25 -n 1 -G 4 -o data/Pten1.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Pten1/Pten1.fa' kmer_count -l 25 -n 1 -G 4 -o data/Pten2.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Pten2/Pten2.fa' Total: 263520 kmers Total: 281579 kmers Total: 570942 kmers Total: 617703 kmers

Thu Feb 2 00:16:30 2017 kmer_count -l 25 -n 1 -G 4 -o data/Purvi.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Purvi/Purvi.fa' running batch 3/27 kmer_count -l 25 -n 1 -G 4 -o data/Raber.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Raber/Raber.fa' kmer_count -l 25 -n 1 -G 4 -o data/Ralb1.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Ralb1/Ralb1.fa' kmer_count -l 25 -n 1 -G 4 -o data/Ralb2.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Ralb2/Ralb2.fa' Total: 578501 kmers Total: 622151 kmers Total: 897710 kmers Total: 1129223 kmers

Thu Feb 2 00:16:32 2017 running batch 4/27 kmer_count -l 25 -n 1 -G 4 -o data/Rbald.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rbald/Rbald.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rbar1.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rbar1/Rbar1.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rbar2.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rbar2/Rbar2.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rbert.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rbert/Rbert.fa' Total: 853690 kmers Total: 903695 kmers Total: 1105437 kmers Total: 1112646 kmers

Thu Feb 2 00:16:36 2017 running batch 5/27 kmer_count -l 25 -n 1 -G 4 -o data/Rbifl.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rbifl/Rbifl.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rbrac.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rbrac/Rbrac.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rbras.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rbras/Rbras.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rbro1.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rbro1/Rbro1.fa' Total: 1854 kmers Total: 690664 kmers Total: 944905 kmers Total: 1091638 kmers

Thu Feb 2 00:16:38 2017 kmer_count -l 25 -n 1 -G 4 -o data/Rbro2.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rbro2/Rbro2.fa' running batch 6/27 kmer_count -l 25 -n 1 -G 4 -o data/Rbro3.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rbro3/Rbro3.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rbuch.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rbuch/Rbuch.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rcali.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rcali/Rcali.fa' Total: 900477 kmers Total: 1131040 kmers Total: 1277587 kmers Total: 1299140 kmers

Thu Feb 2 00:16:41 2017 running batch 7/27 kmer_count -l 25 -n 1 -G 4 -o data/Rcap1.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rcap1/Rcap1.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rcap2.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rcap2/Rcap2.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rceph.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rceph/Rceph.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rceps.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rceps/Rceps.fa' Total: 882956 kmers Total: 1016400 kmers Total: 1824643 kmers Total: 1824965 kmers

kmer_count -l 25 -n 1 -G 4 -o data/Rchal.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rchal/Rchal.fa' Thu Feb 2 00:16:46 2017 running batch 8/27 kmer_count -l 25 -n 1 -G 4 -o data/Rchap.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rchap/Rchap.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rchin.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rchin/Rchin.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rcili.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rcili/Rcili.fa' Total: 518702 kmers Total: 680853 kmers Total: 853249 kmers Total: 1020510 kmers

kmer_count -l 25 -n 1 -G 4 -o data/Rcilo.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rcilo/Rcilo.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rcolo.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rcolo/Rcolo.fa' Thu Feb 2 00:16:48 2017 running batch 9/27 kmer_count -l 25 -n 1 -G 4 -o data/Rcoma.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rcoma/Rcoma.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rcomp.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rcomp/Rcomp.fa' Total: 522384 kmers Total: 791138 kmers Total: 921074 kmers Total: 1860267 kmers

kmer_count -l 25 -n 1 -G 4 -o data/Rcrin.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rcrin/Rcrin.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rcube.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rcube/Rcube.fa' Thu Feb 2 00:16:52 2017 running batch 10/27 kmer_count -l 25 -n 1 -G 4 -o data/Rdebi.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rdebi/Rdebi.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rdecu.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rdecu/Rdecu.fa' Total: 779288 kmers Total: 1087005 kmers Total: 1342066 kmers Total: 1506813 kmers

Thu Feb 2 00:16:56 2017 running batch 11/27 kmer_count -l 25 -n 1 -G 4 -o data/Rdist.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rdist/Rdist.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rdiva.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rdiva/Rdiva.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rdive.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rdive/Rdive.fa' kmer_count -l 25 -n 1 -G 4 -o data/Relat.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Relat/Relat.fa' Total: 18214 kmers Total: 672527 kmers Total: 956949 kmers Total: 1399766 kmers

kmer_count -l 25 -n 1 -G 4 -o data/Relli.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Relli/Relli.fa' Thu Feb 2 00:16:59 2017 running batch 12/27 kmer_count -l 25 -n 1 -G 4 -o data/Remac.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Remac/Remac.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rexal.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rexal/Rexal.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rexim.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rexim/Rexim.fa' Total: 653168 kmers Total: 583478 kmers Total: 853401 kmers Total: 964250 kmers

kmer_count -l 25 -n 1 -G 4 -o data/Rfasc.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rfasc/Rfasc.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rfern.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rfern/Rfern.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rfili.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rfili/Rfili.fa' Thu Feb 2 00:17:01 2017 running batch 13/27 kmer_count -l 25 -n 1 -G 4 -o data/Rfusc.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rfusc/Rfusc.fa' Total: 486312 kmers Total: 645445 kmers Total: 756005 kmers Total: 946328 kmers

kmer_count -l 25 -n 1 -G 4 -o data/Rgale.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rgale/Rgale.fa' Thu Feb 2 00:17:03 2017 running batch 14/27 kmer_count -l 25 -n 1 -G 4 -o data/Rglaz.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rglaz/Rglaz.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rglo1.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rglo1/Rglo1.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rglo2.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rglo2/Rglo2.fa' Total: 340459 kmers Total: 790820 kmers Total: 843002 kmers Total: 1631524 kmers

Thu Feb 2 00:17:07 2017 running batch 15/27 kmer_count -l 25 -n 1 -G 4 -o data/Rglo3.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rglo3/Rglo3.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rgray.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rgray/Rgray.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rharp.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rharp/Rharp.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rharv.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rharv/Rharv.fa' Total: 715601 kmers Total: 914590 kmers Total: 1038797 kmers Total: 1184725 kmers

kmer_count -l 25 -n 1 -G 4 -o data/Rimer.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rimer/Rimer.fa' Thu Feb 2 00:17:10 2017 running batch 16/27 kmer_count -l 25 -n 1 -G 4 -o data/Rinex.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rinex/Rinex.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rinte.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rinte/Rinte.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rknie.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rknie/Rknie.fa' Total: 702146 kmers Total: 1030774 kmers Total: 1347309 kmers Total: 1341808 kmers

kmer_count -l 25 -n 1 -G 4 -o data/Rlati.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rlati/Rlati.fa' Thu Feb 2 00:17:13 2017 running batch 17/27 kmer_count -l 25 -n 1 -G 4 -o data/Rlava.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rlava/Rlava.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rlept.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rlept/Rlept.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rmacr.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rmacr/Rmacr.fa' Total: 525570 kmers Total: 576172 kmers Total: 1345604 kmers Total: 1596797 kmers

Thu Feb 2 00:17:17 2017 kmer_count -l 25 -n 1 -G 4 -o data/Rmari.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rmari/Rmari.fa' running batch 18/27 kmer_count -l 25 -n 1 -G 4 -o data/Rmarl.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rmarl/Rmarl.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rmega.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rmega/Rmega.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rmegp.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rmegp/Rmegp.fa' Total: 646760 kmers Total: 589472 kmers Total: 905129 kmers Total: 1478663 kmers

Thu Feb 2 00:17:20 2017 running batch 19/27 kmer_count -l 25 -n 1 -G 4 -o data/Rmicp.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rmicp/Rmicp.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rmicr.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rmicr/Rmicr.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rmili.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rmili/Rmili.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rmixt.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rmixt/Rmixt.fa' Total: 369315 kmers Total: 971278 kmers Total: 1700172 kmers Total: 1870676 kmers

Thu Feb 2 00:17:25 2017 running batch 20/27 kmer_count -l 25 -n 1 -G 4 -o data/Rnite.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rnite/Rnite.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rnive.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rnive/Rnive.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rodor.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rodor/Rodor.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rpall.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rpall/Rpall.fa' Total: 712578 kmers Total: 866813 kmers Total: 903405 kmers Total: 1044035 kmers

Thu Feb 2 00:17:27 2017 kmer_count -l 25 -n 1 -G 4 -o data/Rperp.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rperp/Rperp.fa' running batch 21/27 kmer_count -l 25 -n 1 -G 4 -o data/Rplum.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rplum/Rplum.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rpoly.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rpoly/Rpoly.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rprui.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rprui/Rprui.fa' Total: 8314 kmers Total: 413046 kmers Total: 1192931 kmers Total: 1944980 kmers

Thu Feb 2 00:17:31 2017 running batch 22/27 kmer_count -l 25 -n 1 -G 4 -o data/Rpube.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rpube/Rpube.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rpunc.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rpunc/Rpunc.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rpusi.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rpusi/Rpusi.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rrace.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rrace/Rrace.fa' Total: 1423 kmers Total: 407698 kmers Total: 643732 kmers Total: 1482929 kmers

Thu Feb 2 00:17:34 2017 running batch 23/27 kmer_count -l 25 -n 1 -G 4 -o data/Rradi.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rradi/Rradi.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rrari.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rrari/Rrari.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rreco.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rreco/Rreco.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rried.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rried/Rried.fa' Total: 47773 kmers Total: 679416 kmers Total: 769224 kmers Total: 864651 kmers

Thu Feb 2 00:17:36 2017 running batch 24/27 kmer_count -l 25 -n 1 -G 4 -o data/Rripa.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rripa/Rripa.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rrobu.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rrobu/Rrobu.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rrubr.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rrubr/Rrubr.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rschi.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rschi/Rschi.fa' Total: 19780 kmers Total: 54908 kmers Total: 824393 kmers Total: 903644 kmers

Thu Feb 2 00:17:38 2017 running batch 25/27 kmer_count -l 25 -n 1 -G 4 -o data/Rscir.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rscir/Rscir.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rsesl.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rsesl/Rsesl.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rsier.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rsier/Rsier.fa' kmer_count -l 25 -n 1 -G 4 -o data/RspI1.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/RspI1/RspI1.fa' Total: 696521 kmers Total: 697564 kmers Total: 862815 kmers Total: 812405 kmers

Thu Feb 2 00:17:41 2017 kmer_count -l 25 -n 1 -G 4 -o data/Rsten.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rsten/Rsten.fa' running batch 26/27 kmer_count -l 25 -n 1 -G 4 -o data/Rsulc.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rsulc/Rsulc.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rtene.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rtene/Rtene.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rtenu.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rtenu/Rtenu.fa' Total: 163323 kmers Total: 238938 kmers Total: 1473427 kmers Total: 1624038 kmers

kmer_count -l 25 -n 1 -G 4 -o data/Rtrac.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rtrac/Rtrac.fa' kmer_count -l 25 -n 1 -G 4 -o data/Rvulc.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rvulc/Rvulc.fa' Thu Feb 2 00:17:44 2017 running last batch kmer_count -l 25 -n 1 -G 4 -o data/Rwrig.pkdat.gz -i '/home/rgufal/Documentos/ANDRE/AAF/boot_Folder/Rwrig/Rwrig.fa' Total: 19111 kmers Total: 39221 kmers Total: 1132628 kmers

Thu Feb 2 00:17:47 2017 kmer_merge -k s -c -d '0' -a 'O,M,F' 'data/Pangu.pkdat.gz' 'data/Pgaud.pkdat.gz' 'data/Pgram.pkdat.gz' 'data/Psp1H.pkdat.gz' 'data/Psp2H.pkdat.gz' 'data/Pstri.pkdat.gz' 'data/Pten1.pkdat.gz' 'data/Pten2.pkdat.gz' 'data/Purvi.pkdat.gz' 'data/Raber.pkdat.gz' 'data/Ralb1.pkdat.gz' 'data/Ralb2.pkdat.gz' 'data/Rbald.pkdat.gz' 'data/Rbar1.pkdat.gz' 'data/Rbar2.pkdat.gz' 'data/Rbert.pkdat.gz' 'data/Rbifl.pkdat.gz' 'data/Rbrac.pkdat.gz' 'data/Rbras.pkdat.gz' 'data/Rbro1.pkdat.gz' 'data/Rbro2.pkdat.gz' 'data/Rbro3.pkdat.gz' 'data/Rbuch.pkdat.gz' 'data/Rcali.pkdat.gz' 'data/Rcap1.pkdat.gz' 'data/Rcap2.pkdat.gz' 'data/Rceph.pkdat.gz' 'data/Rceps.pkdat.gz' 'data/Rchal.pkdat.gz' 'data/Rchap.pkdat.gz' 'data/Rchin.pkdat.gz' 'data/Rcili.pkdat.gz' 'data/Rcilo.pkdat.gz' 'data/Rcolo.pkdat.gz' 'data/Rcoma.pkdat.gz' 'data/Rcomp.pkdat.gz' 'data/Rcrin.pkdat.gz' 'data/Rcube.pkdat.gz' 'data/Rdebi.pkdat.gz' 'data/Rdecu.pkdat.gz' 'data/Rdist.pkdat.gz' 'data/Rdiva.pkdat.gz' 'data/Rdive.pkdat.gz' 'data/Relat.pkdat.gz' 'data/Relli.pkdat.gz' 'data/Remac.pkdat.gz' 'data/Rexal.pkdat.gz' 'data/Rexim.pkdat.gz' 'data/Rfasc.pkdat.gz' 'data/Rfern.pkdat.gz' 'data/Rfili.pkdat.gz' 'data/Rfusc.pkdat.gz' 'data/Rgale.pkdat.gz' 'data/Rglaz.pkdat.gz' 'data/Rglo1.pkdat.gz' 'data/Rglo2.pkdat.gz' 'data/Rglo3.pkdat.gz' 'data/Rgray.pkdat.gz' 'data/Rharp.pkdat.gz' 'data/Rharv.pkdat.gz' 'data/Rimer.pkdat.gz' 'data/Rinex.pkdat.gz' 'data/Rinte.pkdat.gz' 'data/Rknie.pkdat.gz' 'data/Rlati.pkdat.gz' 'data/Rlava.pkdat.gz' 'data/Rlept.pkdat.gz' 'data/Rmacr.pkdat.gz' 'data/Rmari.pkdat.gz' 'data/Rmarl.pkdat.gz' 'data/Rmega.pkdat.gz' 'data/Rmegp.pkdat.gz' 'data/Rmicp.pkdat.gz' 'data/Rmicr.pkdat.gz' 'data/Rmili.pkdat.gz' 'data/Rmixt.pkdat.gz' 'data/Rnite.pkdat.gz' 'data/Rnive.pkdat.gz' 'data/Rodor.pkdat.gz' 'data/Rpall.pkdat.gz' 'data/Rperp.pkdat.gz' 'data/Rplum.pkdat.gz' 'data/Rpoly.pkdat.gz' 'data/Rprui.pkdat.gz' 'data/Rpube.pkdat.gz' 'data/Rpunc.pkdat.gz' 'data/Rpusi.pkdat.gz' 'data/Rrace.pkdat.gz' 'data/Rradi.pkdat.gz' 'data/Rrari.pkdat.gz' 'data/Rreco.pkdat.gz' 'data/Rried.pkdat.gz' 'data/Rripa.pkdat.gz' 'data/Rrobu.pkdat.gz' 'data/Rrubr.pkdat.gz' 'data/Rschi.pkdat.gz' 'data/Rscir.pkdat.gz' 'data/Rsesl.pkdat.gz' 'data/Rsier.pkdat.gz' 'data/RspI1.pkdat.gz' 'data/Rsten.pkdat.gz' 'data/Rsulc.pkdat.gz' 'data/Rtene.pkdat.gz' 'data/Rtenu.pkdat.gz' 'data/Rtrac.pkdat.gz' 'data/Rvulc.pkdat.gz' 'data/Rwrig.pkdat.gz' | cut -f 2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78,80,82,84,86,88,90,92,94,96,98,100,102,104,106,108,110,112,114,116,118,120,122,124,126,128,130,132,134,136,138,140,142,144,146,148,150,152,154,156,158,160,162,164,166,168,170,172,174,176,178,180,182,184,186,188,190,192,194,196,198,200,202,204,206,208,210,212,214 | gzip > data/phylokmer.dat.gz Thu Feb 2 00:25:49 2017

Thu Feb 2 00:25:49 2017 start calulation distances . Traceback (most recent call last): File "BetaVersion/nonparametric_bootstrap.py", line 376, in aaf_distance(outFile,nThreads,memory,samples,options.kLen) File "BetaVersion/nonparametric_bootstrap.py", line 92, in aaf_distance total,shared = job.get() File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get raise self._value IOError: bad message length

Thanks, André

fanhuan commented 7 years ago

Hi André,

Sorry for the late reply. I am not able to reproduce your problem. If you're still interested in using AAF, you could consider the step2 only bootstrap if your data has a coverage > 8X. If not, Multiprocessing is not a very user-friendly module and I am planning to replace it with concurrent.futures. I will let you know once that is done and it should greatly help with issues like this.

dabitz commented 7 years ago

Hi Huan,

Thanks for your reply. As you suggested I was able to run this analysis with the step2 only script. It worked fine. However, I am having another issue now that I think is due to my computational resources, because I am unable to finish the distance script, but I was able to run the phylokmer script. Thus, I am not getting the final tree. I think I have too many OTUs (>900) in this run, in contrast to the other one (130 OTUs) which worked fine. I am running in a 32gb RAM computer. Do you think it can be a limitation due to computational resource? Would you be able to run the distance script if I provide you with my phylokmer.dat.gz and phylokmer.wc files?

Best, André

fanhuan commented 7 years ago

Hi André,

Does the distance script stuck with the tree building step? Or it just won't move to the next batch of jobs?

dabitz commented 7 years ago

It keeps on this screen...

André Marques PhD in Plant Biology Postdoctoral fellow Laboratory of Genetic Resources Federal University of Alagoas - Brazil

Em 2 de out de 2017 19:46, "Huan Fan" notifications@github.com escreveu:

Hi André,

Does the distance script stuck with the tree building step? Or it just won't move to the next batch of jobs?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/fanhuan/AAF/issues/1#issuecomment-333686762, or mute the thread https://github.com/notifications/unsubscribe-auth/AYU_o8EL9awaxexY7XKA_xNW-rqwmN1Bks5soWfQgaJpZM4L1S5X .

fanhuan commented 7 years ago

Hi André, was there supposed to be a screenshot? You could send it to me via huan.fan at wisc.edu if you are having problem posting it on Github.

fanhuan commented 7 years ago

Hi André,

Based on your screenshot, I would suggest you use a smaller G, say 1, because when the G is too large, each job is heavy and it takes long to move to the next batch. Actually I would always suggest you use 1G so things move faster. Unlike aaf_phylokmer.py, aaf_distance.py is not memory-intensive. Instead, increasing the number of threads would help a great deal if you have more cores. Your first batch of jobs took over 3h, so I would expect the second batch to take similar amount of time. Does your job just get stuck like that after more than 4 or 5 hours? Are you using a local machine or a remote one? If you are using a remote one, I would suggest using nohup. Let me know if I am not making sense.

dabitz commented 7 years ago

Hi Huan, Many thanks for your suggestion. I will try using a smaller G. In fact it goes to to next batches, but as it is taking a lot of time for each one I was unable so far to finish it. I am using a local machine. Best, André

2017-10-03 11:34 GMT-03:00 Huan Fan notifications@github.com:

Hi André,

Based on your screenshot, I would suggest you use a smaller G, say 1, because when the G is too large, each job is heavy and it takes long to move to the next batch. Actually I would always suggest you use 1G so things move faster. Unlike aaf_phylokmer.py, aaf_distance.py is not memory-intensive. Instead, increasing the number of threads would help a great deal if you have more cores. Your first batch of jobs took over 3h, so I would expect the second batch to take similar amount of time. Does your job just get stuck like that after more than 4 or 5 hours? Are you using a local machine or a remote one? If you are using a remote one, I would suggest using nohup. Let me know if I am not making sense.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/fanhuan/AAF/issues/1#issuecomment-333861668, or mute the thread https://github.com/notifications/unsubscribe-auth/AYU_o0qbjLyujyGEIFw8cSX3Eal2t-qLks5sokXxgaJpZM4L1S5X .

fanhuan commented 7 years ago

Let me know how it goes. If it is still taking as long, you can try to increase the number of threads even if you don't have more cores, however I don't think this would help much since there isn't much idle time, but might increase the waiting time for all jobs to finish. 4 threads are very humble in front of >900 samples... The most I've done was with ~1000 samples. I had a machine with 32 cores so the distance calculation part was fine. However the tree construction part took 3 days or so.

dabitz commented 7 years ago

Do you think it is good to run the python script in parallel? In this case I would be able to run with 12 cores and 80Gb RAM.

2017-10-03 11:46 GMT-03:00 Huan Fan notifications@github.com:

Let me know how it goes. If it is still taking as long, you can try to increase the number of threads even if you don't have more cores, however I don't think this would help much since there isn't much idle time, but might increase the waiting time for all jobs to finish. 4 threads are very humble in front of >900 samples... The most I've done was with ~1000 samples. I had a machine with 32 cores so the distance calculation part was fine. However the tree construction part took 3 days or so.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/fanhuan/AAF/issues/1#issuecomment-333865418, or mute the thread https://github.com/notifications/unsubscribe-auth/AYU_o5QmKNKASDFhvPwSxO2dfOzzYAJrks5soki6gaJpZM4L1S5X .

azizebrahimi commented 3 years ago

Hi Fanhuan, I used AAF for short reads .fastq file. I used R1 and R2. For chloroplast genome I have .fasta file. Have you used AAF to run create phylogeny with fasta file?