search

alekseyzimin / masurca

GNU General Public License v3.0
242 stars 35 forks source link

Unitig failure in CA #13

Open yyx8671 opened 6 years ago

yyx8671 commented 6 years ago

Hi, I was using masurca to assemble a genome. Unfortunately, I got an error which I cannot fix, can you please help to fix it. 

cat: write error: Broken pipe
mkdir: cannot create directory `CA/fix_unitig_consensus': File exists
INSERTING unitig 82475
MultiAlignStore::dumpMASRfile()-- Writing '../genome.tigStore/seqDB.v001.p005.utg' partitioned.
INSERTING unitig 151276
MultiAlignStore::dumpMASRfile()-- Writing '../genome.tigStore/seqDB.v001.p008.utg' partitioned.

In runCA3.out, it says

ERROR: Failed with signal SEGV (11)
================================================================================

runCA failed.

----------------------------------------
Stack trace:

 at /gpfs1/sw1/Projects/MaSuRCA/3.2.4/bin/../CA/Linux-amd64/bin/runCA line 1121.
        main::caFailure("scaffolder failed", "/30days/GROUPS/Q0196RW/yyuan/myProject/masurca_20+30/"...) called at /gpfs1/sw1/Projects/MaSuRCA/3.2.4/bin/../CA/Linux-amd64/bin/runCA line 4066
        main::CGW("7-0-CGW", undef, "/30days/GROUPS/Q0196RW/yyuan/myProject/masurca_20+30/"..., 2, undef, 1) called at /gpfs1/sw1/Projects/MaSuRCA/3.2.4/bin/../CA/Linux-amd64/bin/runCA line 4260
        main::scaffolder() called at /gpfs1/sw1/Projects/MaSuRCA/3.2.4/bin/../CA/Linux-amd64/bin/runCA line 5346

----------------------------------------
Last few lines of the relevant log file (/30days/GROUPS/Q0196RW/yyuan/myProject/masurca_20+30/CA/7-0-CGW/cgw.out):

...processed 61000000 fragments.
...processed 62000000 fragments.
...processed 63000000 fragments.
...processed 64000000 fragments.
...processed 65000000 fragments.
...processed 66000000 fragments.
...processed 67000000 fragments.
...processed 68000000 fragments.
...processed 69000000 fragments.
...processed 70000000 fragments.
...processed 71000000 fragments.
...processed 72000000 fragments.
...processed 73000000 fragments.
...processed 74000000 fragments.
...processed 75000000 fragments.
...processed 76000000 fragments.
...processed 77000000 fragments.
...processed 78000000 fragments.
Reading unitigs.
ERROR:  Unitig 82475 has no placement; probably not run through consensus.

----------------------------------------
Failure message:

scaffolder failed

I also looked at http://wgs-assembler.sourceforge.net/wiki/index.php/Unitig_Consensus_Failures_in_CA_6. However, it didn't give a clear solution.

alekseyzimin commented 6 years ago

erase the 4-unitigger 5-, 7- and genome.tigStore and try the latest 3.2.6b version ftp://ftp.genome.umd.edu/pub/MaSuRCA/beta/MaSuRCA-3.2.6b.tar.gz

alekseyzimin commented 6 years ago

please use the latest 3.2.6 version -- the unitig consensus is fixed there

MCH74 commented 6 years ago

Hi, a couple of unitig consensus jobs still failed during runCA1, even with the 3.2.6b version. Have there been changes in the full version 3.2.6 compared to this beta version? Thanks, Mark

alekseyzimin commented 6 years ago

Yes, there have been, specifically in the unitig consensus. Please use the official 3.2.6 version.

--Aleksey

On Thu, May 10, 2018 at 3:58 AM, MarkChristianHarrison < notifications@github.com> wrote:

Hi, a couple of unitig consensus jobs still failed during runCA1, even with the 3.2.6b version. Have there been changes in the full version 3.2.6 compared to this beta version? Thanks, Mark

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/alekseyzimin/masurca/issues/13#issuecomment-387981739, or mute the thread https://github.com/notifications/unsubscribe-auth/AZ9zHdumAgYl9sdV-vf8DZWnnxzfv_x7ks5tw_MjgaJpZM4TEj65 .

-- Dr. Alexey V. Zimin Associate Research Scientist Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA (301)-437-6260 http://www.genome.umd.edu http://masurca.blogspot.com

gorliver commented 6 years ago

Hi, I used the official 3.2.6 version and have the same issue. Here is the last few lines of cgw.out: ...processed 44000000 fragments. ...processed 45000000 fragments. ...processed 46000000 fragments. ...processed 47000000 fragments. ...processed 48000000 fragments. ...processed 49000000 fragments. ...processed 50000000 fragments. ...processed 51000000 fragments. ...processed 52000000 fragments. ...processed 53000000 fragments. ...processed 54000000 fragments. ...processed 55000000 fragments. ...processed 56000000 fragments. Reading unitigs. ERROR: Unitig 3856 has no placement; probably not run through consensus.

Failure message:

scaffolder failed

sunnycqcn commented 6 years ago

Hello, How much your genome size and how much data? I used MaSuRCA which took more than half year with 20 cpus. Then I stopped it. Thanks

发自我的 iPhone

在 2018年6月21日，下午1:22，gorliver notifications@github.com 写道：

Hi, I used the official 3.2.6 version and have the same issue. Here is the last few lines of cgw.out: ...processed 44000000 fragments. ...processed 45000000 fragments. ...processed 46000000 fragments. ...processed 47000000 fragments. ...processed 48000000 fragments. ...processed 49000000 fragments. ...processed 50000000 fragments. ...processed 51000000 fragments. ...processed 52000000 fragments. ...processed 53000000 fragments. ...processed 54000000 fragments. ...processed 55000000 fragments. ...processed 56000000 fragments. Reading unitigs. ERROR: Unitig 3856 has no placement; probably not run through consensus.

Failure message:

scaffolder failed

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

sguizard commented 6 years ago

Hi,

I'm using the version 3.2.6 of masurca and I got the same error. Did someone found a solution for fixing this ?

Thanks in advance. Regards

cat runCA3.out 
#
#  Reading options from '/some...path/runCA.spec'
#
computeInsertSize=0
gkpFixInsertSizes=0
unitigger=bog
cnsConcurrency=22
gkpFixInsertSizes=0
jellyfishHashSize=6181105292
ovlRefBlockSize=11779065 
ovlHashBlockSize=1177906
ovlCorrBatchSize=1177906
ovlMerThreshold=75
merylThreads=22
merylMemory=16384
ovlStoreMemory=16384
ovlThreads=1
frgCorrThreads=1
frgCorrConcurrency=22 
ovlCorrConcurrency=22 
ovlConcurrency=22
#
#  Reading options from the command line.
#
cgwErrorRate=0.15
doFragmentCorrection=0
doOverlapBasedTrimming=0
doExtendClearRanges=0
ovlMerSize=30
ovlMemory=4GB
----------------------------------------START Wed Jul 11 02:12:23 2018
/usr/local/CA/Linux-amd64/bin/cgw  \
  -j 1 -k 5 \
  -r 5 \
  -s 2 \
  -z \
  -P 2 \
  -B 920239 \
  -m 100 \
  -g /some...path/masurca/CA/genome.gkpStore \
  -t /some...path/masurca/CA/genome.tigStore \
  -o /some...path/masurca/CA/7-0-CGW/genome \
 > /some...path/masurca/CA/7-0-CGW/cgw.out 2>&1
----------------------------------------END Wed Jul 11 02:14:04 2018 (101 seconds)
ERROR: Failed with signal SEGV (11)
================================================================================

runCA failed.

----------------------------------------
Stack trace:

 at /usr/local/bin/../CA/Linux-amd64/bin/runCA line 1121.
    main::caFailure("scaffolder failed", "/some...path/"...) called at /usr/local/bin/../CA/Linux-amd64/bin/runCA line 4066
    main::CGW("7-0-CGW", undef, "/some...path/"..., 2, undef, 1) called at /usr/local/bin/../CA/Linux-amd64/bin/runCA line 4260
    main::scaffolder() called at /usr/local/bin/../CA/Linux-amd64/bin/runCA line 5346

----------------------------------------
Last few lines of the relevant log file (/some...path/masurca/CA/7-0-CGW/cgw.out):

...processed 102000000 fragments.
...processed 103000000 fragments.
...processed 104000000 fragments.
...processed 105000000 fragments.
...processed 106000000 fragments.
...processed 107000000 fragments.
...processed 108000000 fragments.
...processed 109000000 fragments.
...processed 110000000 fragments.
...processed 111000000 fragments.
...processed 112000000 fragments.
...processed 113000000 fragments.
...processed 114000000 fragments.
...processed 115000000 fragments.
...processed 116000000 fragments.
...processed 117000000 fragments.
Reading unitigs.
...processed 100000 unitigs.
ERROR:  Unitig 117779 has no placement; probably not run through consensus.
Segmentation fault (core dumped)

----------------------------------------
Failure message:

scaffolder failed
evolscientist commented 6 years ago

I'm having the same issue using 3.2.7. If there is a workaround, I'm happy to try fixes.

Thanks,

#

Reading options from '/nobackup/rogers_research/Bivalves/Ehopetonensis/runCA.spec'

# computeInsertSize=0 gkpFixInsertSizes=0 unitigger=bog cnsConcurrency=32 gkpFixInsertSizes=0 jellyfishHashSize=8000000000 ovlRefBlockSize=12559593 ovlHashBlockSize=1255959 ovlCorrBatchSize=1255959 ovlMerThreshold=75 merylThreads=32 merylMemory=16384 ovlStoreMemory=16384 ovlThreads=1 frgCorrThreads=1 frgCorrConcurrency=32 ovlCorrConcurrency=32 ovlConcurrency=32 #

Reading options from the command line.

# cgwErrorRate=0.15 doFragmentCorrection=0 doOverlapBasedTrimming=0 doExtendClearRanges=0 ovlMerSize=30 ovlMemory=4GB ----------------------------------------START Sat Aug 25 07:48:12 2018 /apps/pkg/masurca-3.2.7/rhel7_u2-x86_64/gnu/CA/Linux-amd64/bin/cgw \ -j 1 -k 5 \ -r 5 \ -s 2 \ -z \ -P 2 \ -B 981218 \ -m 100 \ -g /nobackup/rogers_research/Bivalves/Ehopetonensis/CA/genome.gkpStore \ -t /nobackup/rogers_research/Bivalves/Ehopetonensis/CA/genome.tigStore \ -o /nobackup/rogers_research/Bivalves/Ehopetonensis/CA/7-0-CGW/genome \

/nobackup/rogers_research/Bivalves/Ehopetonensis/CA/7-0-CGW/cgw.out 2>&1 sh: line 11: 242994 Segmentation fault (core dumped) /apps/pkg/masurca-3.2.7/rhel7_u2-x86_64/gnu/CA/Linux-amd64/bin/cgw -j 1 -k 5 -r 5 -s 2 -z -P 2 -B 981218 -m 100 -g /nobackup/rogers_research/Bivalves/Ehopetonensis/CA/genome.gkpStore -t /nobackup/rogers_research/Bivalves/Ehopetonensis/CA/genome.tigStore -o /nobackup/rogers_research/Bivalves/Ehopetonensis/CA/7-0-CGW/genome > /nobackup/rogers_research/Bivalves/Ehopetonensis/CA/7-0-CGW/cgw.out 2>&1 ----------------------------------------END Sat Aug 25 07:50:10 2018 (118 seconds) ERROR: Failed with signal SEGV (11)

runCA failed.

Stack trace:

at /apps/pkg/masurca-3.2.7/rhel7_u2-x86_64/gnu/bin/../CA/Linux-amd64/bin/runCA line 1121. main::caFailure("scaffolder failed", "/nobackup/rogers_research/Bivalves/Ehopetonensis/CA/7-0-CGW/c"...) called at /apps/pkg/masurca-3.2.7/rhel7_u2-x86_64/gnu/bin/../CA/Linux-amd64/bin/runCA line 4066 main::CGW("7-0-CGW", undef, "/nobackup/rogers_research/Bivalves/Ehopetonensis/CA/genome.ti"..., 2, undef, 1) called at /apps/pkg/masurca-3.2.7/rhel7_u2-x86_64/gnu/bin/../CA/Linux-amd64/bin/runCA line 4260 main::scaffolder() called at /apps/pkg/masurca-3.2.7/rhel7_u2-x86_64/gnu/bin/../CA/Linux-amd64/bin/runCA line 5346

Last few lines of the relevant log file (/nobackup/rogers_research/Bivalves/Ehopetonensis/CA/7-0-CGW/cgw.out):

...processed 108000000 fragments. ...processed 109000000 fragments. ...processed 110000000 fragments. ...processed 111000000 fragments. ...processed 112000000 fragments. ...processed 113000000 fragments. ...processed 114000000 fragments. ...processed 115000000 fragments. ...processed 116000000 fragments. ...processed 117000000 fragments. ...processed 118000000 fragments. ...processed 119000000 fragments. ...processed 120000000 fragments. ...processed 121000000 fragments. ...processed 122000000 fragments. ...processed 123000000 fragments. ...processed 124000000 fragments. ...processed 125000000 fragments. Reading unitigs. ERROR: Unitig 86924 has no placement; probably not run through consensus.

Failure message:

scaffolder failed

EmelineFavreau commented 6 years ago

Hello, I am having the same issue, using the last version MaSuRCA-3.2.7. Any ideas on how to fix it? Thanks!

computeInsertSize=0
gkpFixInsertSizes=0
unitigger=bog
cnsConcurrency=30
gkpFixInsertSizes=0
jellyfishHashSize=16500000000
ovlRefBlockSize=3855509 
ovlHashBlockSize=385550
ovlCorrBatchSize=385550
ovlMerThreshold=75
merylThreads=30
merylMemory=16384
ovlStoreMemory=16384
ovlThreads=1
frgCorrThreads=1
frgCorrConcurrency=30 
ovlCorrConcurrency=30 
ovlConcurrency=30
#
#  Reading options from the command line.
#
cgwErrorRate=0.15
doFragmentCorrection=0
doOverlapBasedTrimming=0
doExtendClearRanges=0
ovlMerSize=30
ovlMemory=4GB
----------------------------------------START Sun Aug 19 18:27:57 2018
/data/SBCS-WurmLab/software/MaSuRCA-3.2.7/CA/Linux-amd64/bin/cgw  \
  -j 1 -k 5 \
  -r 5 \
  -s 2 \
  -z \
  -P 2 \
  -B 301211 \
  -m 100 \
  -g /data/home/btx077/weekly_autoScratch/2018-08-17-assembling-3P-pheidole-genome/CA/genome.gkpStore \
  -t /data/home/btx077/weekly_autoScratch/2018-08-17-assembling-3P-pheidole-genome/CA/genome.tigStore \
  -o /data/home/btx077/weekly_autoScratch/2018-08-17-assembling-3P-pheidole-genome/CA/7-0-CGW/genome \
 > /data/home/btx077/weekly_autoScratch/2018-08-17-assembling-3P-pheidole-genome/CA/7-0-CGW/cgw.out 2>&1
sh: line 11: 22172 Segmentation fault      /data/SBCS-WurmLab/software/MaSuRCA-3.2.7/CA/Linux-amd64/bin/cgw -j 1 -k 5 -r 5 -s 2 -z -P 2 -B 301211 -m 100 -g /data/home/btx077/weekly_autoScratch/2018-08-17-assembling-3P-pheidole-genome/CA/genome.gkpStore -t /data/home/btx077/weekly_autoScratch/2018-08-17-assembling-3P-pheidole-genome/CA/genome.tigStore -o /data/home/btx077/weekly_autoScratch/2018-08-17-assembling-3P-pheidole-genome/CA/7-0-CGW/genome > /data/home/btx077/weekly_autoScratch/2018-08-17-assembling-3P-pheidole-genome/CA/7-0-CGW/cgw.out 2>&1
----------------------------------------END Sun Aug 19 18:28:08 2018 (11 seconds)
ERROR: Failed with signal SEGV (11)
================================================================================

runCA failed.

----------------------------------------
Stack trace:

 at /data/SBCS-WurmLab/software/MaSuRCA-3.2.7/bin/../CA/Linux-amd64/bin/runCA line 1121.
        main::caFailure('scaffolder failed', '/data/home/btx077/weekly_autoScratch/2018-08-17-assembling-3P...') called at /data/SBCS-WurmLab/software/MaSuRCA-3.2.7/bin/../CA/Linux-amd64/bin/runCA line 4066
        main::CGW('7-0-CGW', undef, '/data/home/btx077/weekly_autoScratch/2018-08-17-assembling-3P...', 2, undef, 1) called at /data/SBCS-WurmLab/software/MaSuRCA-3.2.7/bin/../CA/Linux-amd64/bin/runCA line 4260
        main::scaffolder() called at /data/SBCS-WurmLab/software/MaSuRCA-3.2.7/bin/../CA/Linux-amd64/bin/runCA line 5346

----------------------------------------
Last few lines of the relevant log file (/data/home/btx077/weekly_autoScratch/2018-08-17-assembling-3P-pheidole-genome/CA/7-0-CGW/cgw.out):

...processed 21000000 fragments.
...processed 22000000 fragments.
...processed 23000000 fragments.
...processed 24000000 fragments.
...processed 25000000 fragments.
...processed 26000000 fragments.
...processed 27000000 fragments.
...processed 28000000 fragments.
...processed 29000000 fragments.
...processed 30000000 fragments.
...processed 31000000 fragments.
...processed 32000000 fragments.
...processed 33000000 fragments.
...processed 34000000 fragments.
...processed 35000000 fragments.
...processed 36000000 fragments.
...processed 37000000 fragments.
...processed 38000000 fragments.
Reading unitigs.
ERROR:  Unitig 0 has no placement; probably not run through consensus.

----------------------------------------
Failure message:

scaffolder failed