search

alekseyzimin / masurca

GNU General Public License v3.0
242 stars 35 forks source link

ERROR: Failed with signal ABRT (6) during 0-overlaptrim-overlap stage #82

Open TimothyStephens opened 5 years ago

TimothyStephens commented 5 years ago

Hi,

Sorry to keep posting issues.

I am getting the below error during the end of the 0-overlaptrim-overlap stage. I tried recompiling but this has not helped.

I am not sure if this is related but in the 0-overlaptrim/genome.obtStore.BUILDING directory the tmp.sort.000 file is 556Gigs while the other files are only 16G.

Thanks, Tim.

----------------------------------------START Thu Nov 22 09:36:09 2018 find -L /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim-overlap ( -name *ovb.gz -or -name *ovb ) -print > /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/C CMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim/genome.obtStore.list ----------------------------------------END Thu Nov 22 09:36:19 2018 (10 seconds) ----------------------------------------START Thu Nov 22 09:36:19 2018 /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/CA8/Linux-amd64/bin/overlapStoreBuild -obt -o /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim/genom e.obtStore.BUILDING -g /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/genome.gkpStore -M 65536 -L /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLI ES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim/genome.obtStore.list > /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim/genome.obtStore.err 2>&1 sh: line 1: 2242 Aborted /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/CA8/Linux-amd64/bin/overlapStoreBuild -obt -o /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/ CA.mr.41.15.17.0.029/0-overlaptrim/genome.obtStore.BUILDING -g /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/genome.gkpStore -M 65536 -L /mnt/beegfs/timothy.stephens/POLARE LLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim/genome.obtStore.list > /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-ov erlaptrim/genome.obtStore.err 2>&1 ----------------------------------------END Thu Nov 22 12:33:35 2018 (10636 seconds) ERROR: Failed with signal ABRT (6)

runCA failed.

Stack trace:

at /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/bin/../CA8/Linux-amd64/bin/runCA line 1613, line 50338. main::caFailure('failed to build the obt store', '/mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GEN...') called at /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/bin/../CA8/Linux-amd64/bin/runCA line 4081 main::overlapTrim() called at /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/bin/../CA8/Linux-amd64/bin/runCA line 6522

Last few lines of the relevant log file (/mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim/genome.obtStore.err):

bucketizing /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim-overlap/051/050330.ovb.gz bucketizing /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim-overlap/051/050331.ovb.gz bucketizing /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim-overlap/051/050332.ovb.gz bucketizing /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim-overlap/051/050333.ovb.gz bucketizing /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim-overlap/051/050334.ovb.gz bucketizing /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim-overlap/051/050335.ovb.gz bucketizing DONE! overlaps skipped: 10806781738 OBT - low quality 0 DUP - non-duplicate overlap 0 DUP - different library 0 DUP - dedup not requested terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc

Failed with 'Aborted'

Backtrace (mangled):

/gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/CA8/Linux-amd64/bin/overlapStoreBuild[0x405175] /lib64/libpthread.so.0(+0xf5e0)[0x2b091f16e5e0] /lib64/libc.so.6(gsignal+0x37)[0x2b091fe781f7] /lib64/libc.so.6(abort+0x148)[0x2b091fe798e8] /opt/gnu/gcc/lib64/libstdc++.so.6(_ZN9__gnu_cxx27verbose_terminate_handlerEv+0x125)[0x2b091f40c225] /opt/gnu/gcc/lib64/libstdc++.so.6(+0x8eff6)[0x2b091f409ff6] /opt/gnu/gcc/lib64/libstdc++.so.6(+0x8f041)[0x2b091f40a041] /opt/gnu/gcc/lib64/libstdc++.so.6(+0x8f284)[0x2b091f40a284] /opt/gnu/gcc/lib64/libstdc++.so.6(+0x8f77c)[0x2b091f40a77c] /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/CA8/Linux-amd64/bin/overlapStoreBuild[0x40301a] /lib64/libc.so.6(libc_start_main+0xf5)[0x2b091fe64c05] /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/CA8/Linux-amd64/bin/overlapStoreBuild[0x403282]

Backtrace (demangled):

[0] /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/CA8/Linux-amd64/bin/overlapStoreBuild() [0x405175] [1] /lib64/libpthread.so.0::(null) + 0xf5e0 [0x2b091f16e5e0] [2] /lib64/libc.so.6::(null) + 0x37 [0x2b091fe781f7] [3] /lib64/libc.so.6::(null) + 0x148 [0x2b091fe798e8] [4] /opt/gnu/gcc/lib64/libstdc++.so.6::__gnu_cxx::__verbose_terminate_handler() + 0x125 [0x2b091f40c225] [5] /opt/gnu/gcc/lib64/libstdc++.so.6::(null) + 0x8eff6 [0x2b091f409ff6] [6] /opt/gnu/gcc/lib64/libstdc++.so.6::(null) + 0x8f041 [0x2b091f40a041] [7] /opt/gnu/gcc/lib64/libstdc++.so.6::(null) + 0x8f284 [0x2b091f40a284] [8] /opt/gnu/gcc/lib64/libstdc++.so.6::(null) + 0x8f77c [0x2b091f40a77c] [9] /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/CA8/Linux-amd64/bin/overlapStoreBuild() [0x40301a] [10] /lib64/libc.so.6::(null) + 0xf5 [0x2b091fe64c05] [11] /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/CA8/Linux-amd64/bin/overlapStoreBuild() [0x403282]

GDB:

Failure message:

failed to build the obt store

sunnycqcn commented 5 years ago

Hi , If you set ovlMemory=4096, it should work well. Best, Fuyou

Sent from my iPhone

On Nov 21, 2018, at 11:22 PM, TimothyStephens notifications@github.com wrote:

Hi,

Sorry to keep posting issues.

I am getting the below error during the end of the 0-overlaptrim-overlap stage. I tried recompiling but this has not helped.

I am not sure if this is related but in the 0-overlaptrim/genome.obtStore.BUILDING directory the tmp.sort.000 file is 556Gigs while the other files are only 16G.

Thanks, Tim.

----------------------------------------START Thu Nov 22 09:36:09 2018 find -L /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim-overlap ( -name ovb.gz -or -name ovb ) -print > /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/C CMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim/genome.obtStore.list ----------------------------------------END Thu Nov 22 09:36:19 2018 (10 seconds) ----------------------------------------START Thu Nov 22 09:36:19 2018 /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/CA8/Linux-amd64/bin/overlapStoreBuild -obt -o /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim/genom e.obtStore.BUILDING -g /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/genome.gkpStore -M 65536 -L /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLI ES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim/genome.obtStore.list > /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim/genome.obtStore.err 2>&1 sh: line 1: 2242 Aborted /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/CA8/Linux-amd64/bin/overlapStoreBuild -obt -o /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/ CA.mr.41.15.17.0.029/0-overlaptrim/genome.obtStore.BUILDING -g /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/genome.gkpStore -M 65536 -L /mnt/beegfs/timothy.stephens/POLARE LLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim/genome.obtStore.list > /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-ov erlaptrim/genome.obtStore.err 2>&1 ----------------------------------------END Thu Nov 22 12:33:35 2018 (10636 seconds) ERROR: Failed with signal ABRT (6)

runCA failed.

Stack trace:

at /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/bin/../CA8/Linux-amd64/bin/runCA line 1613, line 50338. main::caFailure('failed to build the obt store', '/mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GEN...') called at /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/bin/../CA8/Linux-amd64/bin/runCA line 4081 main::overlapTrim() called at /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/bin/../CA8/Linux-amd64/bin/runCA line 6522

Last few lines of the relevant log file (/mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim/genome.obtStore.err):

bucketizing /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim-overlap/051/050330.ovb.gz bucketizing /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim-overlap/051/050331.ovb.gz bucketizing /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim-overlap/051/050332.ovb.gz bucketizing /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim-overlap/051/050333.ovb.gz bucketizing /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim-overlap/051/050334.ovb.gz bucketizing /mnt/beegfs/timothy.stephens/POLARELLA_GLACIALIS/CCMP1383/GENOME_ASSEMBLIES/MaSuRCA_3.2.8/ASSEMBLY/CA.mr.41.15.17.0.029/0-overlaptrim-overlap/051/050335.ovb.gz bucketizing DONE! overlaps skipped: 10806781738 OBT - low quality 0 DUP - non-duplicate overlap 0 DUP - different library 0 DUP - dedup not requested terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc

Failed with 'Aborted'

Backtrace (mangled):

/gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/CA8/Linux-amd64/bin/overlapStoreBuild[0x405175] /lib64/libpthread.so.0(+0xf5e0)[0x2b091f16e5e0] /lib64/libc.so.6(gsignal+0x37)[0x2b091fe781f7] /lib64/libc.so.6(abort+0x148)[0x2b091fe798e8] /opt/gnu/gcc/lib64/libstdc++.so.6(_ZN9__gnu_cxx27verbose_terminate_handlerEv+0x125)[0x2b091f40c225] /opt/gnu/gcc/lib64/libstdc++.so.6(+0x8eff6)[0x2b091f409ff6] /opt/gnu/gcc/lib64/libstdc++.so.6(+0x8f041)[0x2b091f40a041] /opt/gnu/gcc/lib64/libstdc++.so.6(+0x8f284)[0x2b091f40a284] /opt/gnu/gcc/lib64/libstdc++.so.6(+0x8f77c)[0x2b091f40a77c] /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/CA8/Linux-amd64/bin/overlapStoreBuild[0x40301a] /lib64/libc.so.6(libc_start_main+0xf5)[0x2b091fe64c05] /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/CA8/Linux-amd64/bin/overlapStoreBuild[0x403282]

Backtrace (demangled):

[0] /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/CA8/Linux-amd64/bin/overlapStoreBuild() [0x405175] [1] /lib64/libpthread.so.0::(null) + 0xf5e0 [0x2b091f16e5e0] [2] /lib64/libc.so.6::(null) + 0x37 [0x2b091fe781f7] [3] /lib64/libc.so.6::(null) + 0x148 [0x2b091fe798e8] [4] /opt/gnu/gcc/lib64/libstdc++.so.6::__gnu_cxx::__verbose_terminate_handler() + 0x125 [0x2b091f40c225] [5] /opt/gnu/gcc/lib64/libstdc++.so.6::(null) + 0x8eff6 [0x2b091f409ff6] [6] /opt/gnu/gcc/lib64/libstdc++.so.6::(null) + 0x8f041 [0x2b091f40a041] [7] /opt/gnu/gcc/lib64/libstdc++.so.6::(null) + 0x8f284 [0x2b091f40a284] [8] /opt/gnu/gcc/lib64/libstdc++.so.6::(null) + 0x8f77c [0x2b091f40a77c] [9] /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/CA8/Linux-amd64/bin/overlapStoreBuild() [0x40301a] [10] /lib64/libc.so.6::(null) + 0xf5 [0x2b091fe64c05] [11] /gpfs1/scratch/30days/uqtstep3/PROGRAMS/test/MaSuRCA-3.2.8/CA8/Linux-amd64/bin/overlapStoreBuild() [0x403282]

GDB:

Failure message:

failed to build the obt store

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

TimothyStephens commented 5 years ago

Hi,

The version of the Celera Assembler used by MaSuRCA does not support this parameter any more (see below error). I will try it with the new parameters but it looks like I will have to rerun the overlap commands first.

ERROR: the runCA option ovlMemory is not recognized. This runCA functionality changed in CABOG version 7. Here are suggestions for new runCA option values: ovlHashBits=23 ovlHashBlockLength= 30000000 (replaces ovlMemory=2GB) ovlHashBits=24 ovlHashBlockLength=110000000 (replaces ovlMemory=4GB) ovlHashBits=25 ovlHashBlockLength=180000000 (replaces ovlMemory=8GB)

sunnycqcn commented 5 years ago

hello, you can reset up config file in CA parameter as following: CA_PARAMETERS = ovlStoreMemory=4096 I met the same issue, I changed this. Best, Fuyou

On Thu, Nov 22, 2018 at 6:42 AM TimothyStephens notifications@github.com wrote:

Hi,

The version of the Celera Assembler used by MaSuRCA does not support this parameter any more (see below error). I will try it with the new parameters but it looks like I will have to rerun the overlap commands first.

ERROR: the runCA option ovlMemory is not recognized. This runCA functionality changed in CABOG version 7. Here are suggestions for new runCA option values: ovlHashBits=23 ovlHashBlockLength= 30000000 (replaces ovlMemory=2GB) ovlHashBits=24 ovlHashBlockLength=110000000 (replaces ovlMemory=4GB) ovlHashBits=25 ovlHashBlockLength=180000000 (replaces ovlMemory=8GB)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/alekseyzimin/masurca/issues/82#issuecomment-441019173, or mute the thread https://github.com/notifications/unsubscribe-auth/AXaRKHiAWvt5LE-fcF6zmAiLqqI48AUGks5uxpuqgaJpZM4Yurh7 .

-- Fuyou Fu, Ph.D. Department of Botany and Plant Pathology Purdue University USA

TimothyStephens commented 5 years ago

Thank you!

Adding ovlStoreMemory=4096 has fixed the problem. My assembly is now progressing.

Tim.

sunnycqcn commented 5 years ago

Thanks. I found it is so fast that we do not need mutiple threads. Fuyou

On Fri, Nov 23, 2018 at 12:44 AM TimothyStephens notifications@github.com wrote:

Thank you!

Adding ovlStoreMemory=4096 has fixed the problem. My assembly is now progressing.

Tim.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/alekseyzimin/masurca/issues/82#issuecomment-441165383, or mute the thread https://github.com/notifications/unsubscribe-auth/AXaRKMRBXY6gngi1MJFScM2mQKS7mQawks5ux5lRgaJpZM4Yurh7 .

-- Fuyou Fu, Ph.D. Department of Botany and Plant Pathology Purdue University USA