search

marbl / canu

A single molecule sequence assembler for genomes large and small.
http://canu.readthedocs.io/
654 stars 179 forks source link

Unitigger failed - Segmentation fault #267

Closed freddymonteiro closed 7 years ago

freddymonteiro commented 7 years ago

Hi, I am using canu on two different PacBio ccs datasets. One is assembled without issues, but the second terminates at Unitiging stage. Both unitiging attempts failed. Removing 4-unitigger/ and re-running canu also fails.

Checked https://github.com/marbl/canu/issues/139 and https://github.com/marbl/canu/issues/154 but commits look diferent in my system.

Any suggestion?

Please see below => canu version => Run details => unitigger.err (in 4-unitigger)

=> canu version Canu v1.3 (+218 commits) r7834 0049b3bfc0dbe2f1f6d2dd4d96763ad9a4826202.

=> Run details $ canu -p 9332_mogene_B01 -d 9332_mogene_B01/ genomeSize=2m errorRate=0.01 trimReadsCoverage=2 -pacbio-corrected 9332_mogene_B01.ccs.3pass-Q10.fastq

-- Detected Java(TM) Runtime Environment '1.8.0_91' (from 'java'). -- Detected gnuplot version '4.6 patchlevel 6' (from 'gnuplot') and image format 'png'. -- Detected 8 CPUs and 12 gigabytes of memory.

-- No grid engine detected, grid disabled.

-- Allowed to run 2 jobs concurrently, and use up to 4 compute threads and 6 GB memory for stage 'bogart (unitigger)'. -- Allowed to run 1 job concurrently, and use up to 8 compute threads and 6 GB memory for stage 'mhap (overlapper)'. -- Allowed to run 1 job concurrently, and use up to 8 compute threads and 6 GB memory for stage 'mhap (overlapper)'. -- Allowed to run 1 job concurrently, and use up to 8 compute threads and 6 GB memory for stage 'mhap (overlapper)'. -- Allowed to run 2 jobs concurrently, and use up to 4 compute threads and 2 GB memory for stage 'read error detection (overlap error adjustment)'. -- Allowed to run 8 jobs concurrently, and use up to 1 compute thread and 1 GB memory for stage 'overlap error adjustment'. -- Allowed to run 2 jobs concurrently, and use up to 4 compute threads and 12 GB memory for stage 'utgcns (consensus)'. -- Allowed to run 8 jobs concurrently, and use up to 1 compute thread and 2 GB memory for stage 'overlap store parallel bucketizer'. -- Allowed to run 8 jobs concurrently, and use up to 1 compute thread and 8 GB memory for stage 'overlap store parallel sorting'. -- Allowed to run 8 jobs concurrently, and use up to 1 compute thread and 2 GB memory for stage 'overlapper'. -- Allowed to run 1 job concurrently, and use up to 8 compute threads and 8 GB memory for stage 'overlapper'. -- Allowed to run 1 job concurrently, and use up to 8 compute threads and 8 GB memory for stage 'overlapper'. -- Allowed to run 2 jobs concurrently, and use up to 4 compute threads and 6 GB memory for stage 'meryl (k-mer counting)'. -- Allowed to run 4 jobs concurrently, and use up to 2 compute threads and 6 GB memory for stage 'falcon_sense (read correction)'. -- Allowed to run 1 job concurrently, and use up to 8 compute threads and 6 GB memory for stage 'minimap (overlapper)'. -- Allowed to run 1 job concurrently, and use up to 8 compute threads and 6 GB memory for stage 'minimap (overlapper)'.

-- Allowed to run 1 job concurrently, and use up to 8 compute threads and 6 GB memory for stage 'minimap (overlapper)'.

-- This is canu parallel iteration #1, out of a maximum of 2 attempts.

-- Final error rates before starting pipeline:

-- genomeSize -- 2000000

-- errorRate -- 0.01

-- corOvlErrorRate -- 0.03 -- obtOvlErrorRate -- 0.03

-- utgOvlErrorRate -- 0.03

-- obtErrorRate -- 0.03

-- cnsErrorRate -- 0.03

-- BEGIN TRIMMING

-- Starting command on Mon Oct 10 19:39:27 2016 with 470.542 GB free disk space

/home/freddy/bin/canu/Linux-amd64/bin/gatekeeperCreate \
  -minlength 1000 \
  -o /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.gkpStore.BUILDING \
  /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.gkpStore.gkp \
> /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.gkpStore.BUILDING.err 2>&1

-- Finished on Mon Oct 10 19:39:28 2016 (1 second) with 470.525 GB free disk space

-- In gatekeeper store '/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.gkpStore': -- Found 17475 reads.

-- Found 57460910 bases (28.73 times coverage).

-- Read length histogram (one '' equals 77.7 reads): -- 0 999 0 -- 1000 1999 3362 ****** -- 2000 2999 5439 ** -- 3000 3999 4103 **** -- 4000 4999 1990 * -- 5000 5999 1690 ***** -- 6000 6999 653 **** -- 7000 7999 192 ** -- 8000 8999 40 -- 9000 9999 6

-- Meryl attempt 1 begins.

-- Starting concurrent execution on Mon Oct 10 19:39:28 2016 with 470.525 GB free disk space (1 processes; 2 concurrently)

/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/0-mercounts/meryl.sh 1 > /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/0-mercounts/meryl.000001.out 2>&1

-- Finished on Mon Oct 10 19:39:34 2016 (6 seconds) with 470.476 GB free disk space

-- Meryl finished successfully.

-- Starting command on Mon Oct 10 19:39:34 2016 with 470.476 GB free disk space

/home/freddy/bin/canu/Linux-amd64/bin/meryl \
  -Dh \
  -s /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/0-mercounts/9332_mogene_B01.ms22 \
> /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/0-mercounts/9332_mogene_B01.ms22.histogram \
2> /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/0-mercounts/9332_mogene_B01.ms22.histogram.info

-- Finished on Mon Oct 10 19:39:34 2016 (lickety-split) with 470.476 GB free disk space

-- Starting command on Mon Oct 10 19:39:34 2016 with 470.476 GB free disk space

/home/freddy/bin/canu/Linux-amd64/bin/estimate-mer-threshold  \
  -m /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/0-mercounts/9332_mogene_B01.ms22  \
> /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/0-mercounts/9332_mogene_B01.ms22.estMerThresh.out \
2> /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/0-mercounts/9332_mogene_B01.ms22.estMerThresh.err

-- Finished on Mon Oct 10 19:39:34 2016 (lickety-split) with 470.476 GB free disk space

-- Starting command on Mon Oct 10 19:39:34 2016 with 470.476 GB free disk space

/home/freddy/bin/canu/Linux-amd64/bin/meryl \
  -Dt \
  -n 3972 \
  -s /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/0-mercounts/9332_mogene_B01.ms22 \
> /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/0-mercounts/9332_mogene_B01.ms22.frequentMers.fasta \
2> /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/0-mercounts/9332_mogene_B01.ms22.frequentMers.fasta.err

-- Finished on Mon Oct 10 19:39:34 2016 (lickety-split) with 470.476 GB free disk space

-- Reset obtOvlMerThreshold from auto to 3972.

-- Found 57093935 22-mers; 8660906 distinct and 5384069 unique. Largest count 14004.

-- OVERLAPPER (normal) (trimming) erate=0.03

-- Starting command on Mon Oct 10 19:39:34 2016 with 470.525 GB free disk space

/home/freddy/bin/canu/Linux-amd64/bin/overlapInCorePartition \
 -g  /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.gkpStore \
 -bl 100000000 \
 -bs 0 \
 -rs 2000000 \
 -rl 0 \
 -ol 500 \
 -o  /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/1-overlapper/9332_mogene_B01.partition \
> /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/1-overlapper/9332_mogene_B01.partition.err 2>&1

-- Finished on Mon Oct 10 19:39:34 2016 (lickety-split) with 470.524 GB free disk space

-- Configured 1 overlapInCore jobs.

-- overlapInCore attempt 1 begins with 0 finished, and 1 to compute.

-- Starting concurrent execution on Mon Oct 10 19:39:34 2016 with 470.524 GB free disk space (1 processes; 1 concurrently)

/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/1-overlapper/overlap.sh 1 > /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/1-overlapper/overlap.000001.out 2>&1

-- Finished on Mon Oct 10 19:45:00 2016 (326 seconds) with 470.289 GB free disk space

-- Found 1 overlapInCore output files.

-- overlapInCore compute '/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/1-overlapper': -- kmer hits -- with no overlap 29594726 29594726 +- 0

-- with an overlap 13864172 13864172 +- 0

-- overlaps 13849261 13849261 +- 0 -- contained 0 0 +- 0

-- dovetail 0 0 +- 0

-- overlaps rejected -- multiple per pair 0 0 +- 0 -- bad short window 0 0 +- 0

-- bad long window 0 0 +- 0

-- Starting command on Mon Oct 10 19:45:00 2016 with 470.289 GB free disk space

/home/freddy/bin/canu/Linux-amd64/bin/ovStoreBuild \
 -O /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.ovlStore.BUILDING \
 -G /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.gkpStore \
 -M 2-8 \
 -L /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/1-overlapper/ovljob.files \
 > /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.ovlStore.err 2>&1

-- Finished on Mon Oct 10 19:45:16 2016 (16 seconds) with 469.565 GB free disk space

-- Overlap store '/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.ovlStore' successfully constructed.

-- Purged 0.232 GB in 3 overlap output files and 1 directories.

-- Starting command on Mon Oct 10 19:45:16 2016 with 469.797 GB free disk space

/home/freddy/bin/canu/Linux-amd64/bin/ovStoreStats \
 -G /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.gkpStore \
 -O /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.ovlStore \
 -o /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.ovlStore \
 > /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.ovlStore.summary.err 2>&1

-- Finished on Mon Oct 10 19:45:23 2016 (7 seconds) with 469.797 GB free disk space

-- Overlap store '/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.ovlStore' contains:

-- category reads % read length feature size or coverage analysis

-- middle-missing 2980 17.05 4478.63 +- 1441.17 231.85 +- 452.26 (bad trimming) -- middle-hump 1 0.01 1899.00 +- 0.00 1.00 +- 0.00 (bad trimming) -- no-5-prime 0 0.00 0.00 +- 0.00 0.00 +- 0.00 (bad trimming)

-- no-3-prime 0 0.00 0.00 +- 0.00 0.00 +- 0.00 (bad trimming)

-- low-coverage 763 4.37 3183.63 +- 1220.63 3.37 +- 1.86 (easy to assemble, potential for lower quality consensus) -- unique 1185 6.78 2611.33 +- 1088.53 24.84 +- 8.42 (easy to assemble, perfect, yay) -- repeat-cont 250 1.43 2259.88 +- 848.24 2366.59 +- 1891.95 (potential for consensus errors, no impact on assembly)

-- repeat-dove 715 4.09 2384.78 +- 818.72 1009.24 +- 1495.53 (hard to assemble, likely won't assemble correctly or even at all)

-- span-repeat 3360 19.23 3599.85 +- 1381.37 2235.69 +- 1548.40 (read spans a large repeat, usually easy to assemble) -- uniq-repeat-cont 2258 12.92 2274.08 +- 638.63 (should be uniquely placed, low potential for consensus errors, no impact on assembly) -- uniq-repeat-dove 4533 25.94 2973.70 +- 1141.52 (will end contigs, potential to misassemble)

-- uniq-anchor 822 4.70 4162.71 +- 1182.48 165.38 +- 413.35 (repeat read, with unique section, probable bad read)

-- Starting command on Mon Oct 10 19:45:23 2016 with 469.797 GB free disk space

/home/freddy/bin/canu/Linux-amd64/bin/trimReads \
  -G  /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.gkpStore \
  -O  /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.ovlStore \
  -Co /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/3-overlapbasedtrimming/9332_mogene_B01.1.trimReads.clear \
  -e  0.03 \
  -minlength 1000 \
  -ol 1 \
  -oc 2 \
  -o  /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/3-overlapbasedtrimming/9332_mogene_B01.1.trimReads \
>     /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/3-overlapbasedtrimming/9332_mogene_B01.1.trimReads.err 2>&1

-- Finished on Mon Oct 10 19:45:30 2016 (7 seconds) with 469.795 GB free disk space

-- Starting command on Mon Oct 10 19:45:30 2016 with 469.795 GB free disk space

/home/freddy/bin/canu/Linux-amd64/bin/splitReads \
  -G  /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.gkpStore \
  -O  /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.ovlStore \
  -Ci /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/3-overlapbasedtrimming/9332_mogene_B01.1.trimReads.clear \
  -Co /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/3-overlapbasedtrimming/9332_mogene_B01.2.splitReads.clear \
  -e  0.03 \
  -minlength 1000 \
  -o  /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/3-overlapbasedtrimming/9332_mogene_B01.2.splitReads \
>     /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/3-overlapbasedtrimming/9332_mogene_B01.2.splitReads.err 2>&1

-- Finished on Mon Oct 10 19:45:46 2016 (16 seconds) with 469.224 GB free disk space

-- Starting command on Mon Oct 10 19:45:46 2016 with 469.224 GB free disk space

/home/freddy/bin/canu/Linux-amd64/bin/gatekeeperDumpFASTQ -fasta -nolibname \
  -G /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/9332_mogene_B01.gkpStore \
  -c /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//trimming/3-overlapbasedtrimming/9332_mogene_B01.2.splitReads.clear \
  -o /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//9332_mogene_B01.trimmedReads.gz \
>    /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//9332_mogene_B01.trimmedReads.err 2>&1

-- Finished on Mon Oct 10 19:45:47 2016 (1 second) with 469.212 GB free disk space

-- Trimmed reads saved in '/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//9332_mogene_B01.trimmedReads.fasta.gz'

-- BEGIN ASSEMBLY

-- Starting command on Mon Oct 10 19:45:47 2016 with 469.212 GB free disk space

/home/freddy/bin/canu/Linux-amd64/bin/gatekeeperCreate \
  -minlength 1000 \
  -o /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/9332_mogene_B01.gkpStore.BUILDING \
  /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/9332_mogene_B01.gkpStore.gkp \
> /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/9332_mogene_B01.gkpStore.BUILDING.err 2>&1

-- Finished on Mon Oct 10 19:45:48 2016 (1 second) with 469.199 GB free disk space

-- In gatekeeper store '/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/9332_mogene_B01.gkpStore': -- Found 16148 reads.

-- Found 37674577 bases (18.83 times coverage).

-- Read length histogram (one '' equals 95.07 reads): -- 0 999 0 -- 1000 1999 6597 ****** -- 2000 2999 6655 ** -- 3000 3999 2336 **** -- 4000 4999 309 * -- 5000 5999 206 ** -- 6000 6999 36 -- 7000 7999 8 -- 8000 8999 1

-- Meryl attempt 1 begins.

-- Starting concurrent execution on Mon Oct 10 19:45:48 2016 with 469.198 GB free disk space (1 processes; 2 concurrently)

/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/0-mercounts/meryl.sh 1 > /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/0-mercounts/meryl.000001.out 2>&1

-- Finished on Mon Oct 10 19:45:52 2016 (4 seconds) with 469.174 GB free disk space

-- Meryl finished successfully.

-- Starting command on Mon Oct 10 19:45:52 2016 with 469.174 GB free disk space

/home/freddy/bin/canu/Linux-amd64/bin/meryl \
  -Dh \
  -s /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/0-mercounts/9332_mogene_B01.ms22 \
> /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/0-mercounts/9332_mogene_B01.ms22.histogram \
2> /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/0-mercounts/9332_mogene_B01.ms22.histogram.info

-- Finished on Mon Oct 10 19:45:52 2016 (lickety-split) with 469.174 GB free disk space

-- Starting command on Mon Oct 10 19:45:52 2016 with 469.174 GB free disk space

/home/freddy/bin/canu/Linux-amd64/bin/estimate-mer-threshold  \
  -m /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/0-mercounts/9332_mogene_B01.ms22  \
> /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/0-mercounts/9332_mogene_B01.ms22.estMerThresh.out \
2> /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/0-mercounts/9332_mogene_B01.ms22.estMerThresh.err

-- Finished on Mon Oct 10 19:45:52 2016 (lickety-split) with 469.174 GB free disk space

-- Starting command on Mon Oct 10 19:45:52 2016 with 469.174 GB free disk space

/home/freddy/bin/canu/Linux-amd64/bin/meryl \
  -Dt \
  -n 3849 \
  -s /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/0-mercounts/9332_mogene_B01.ms22 \
> /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/0-mercounts/9332_mogene_B01.ms22.frequentMers.fasta \
2> /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/0-mercounts/9332_mogene_B01.ms22.frequentMers.fasta.err

-- Finished on Mon Oct 10 19:45:52 2016 (lickety-split) with 469.174 GB free disk space

-- Reset utgOvlMerThreshold from auto to 3849.

-- Found 37335469 22-mers; 4213036 distinct and 2389290 unique. Largest count 5025.

-- OVERLAPPER (normal) (assembly) erate=0.03

-- Starting command on Mon Oct 10 19:45:52 2016 with 469.198 GB free disk space

/home/freddy/bin/canu/Linux-amd64/bin/overlapInCorePartition \
 -g  /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/9332_mogene_B01.gkpStore \
 -bl 100000000 \
 -bs 0 \
 -rs 2000000 \
 -rl 0 \
 -ol 500 \
 -o  /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/1-overlapper/9332_mogene_B01.partition \
> /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/1-overlapper/9332_mogene_B01.partition.err 2>&1

-- Finished on Mon Oct 10 19:45:52 2016 (lickety-split) with 469.198 GB free disk space

-- Configured 1 overlapInCore jobs.

-- overlapInCore attempt 1 begins with 0 finished, and 1 to compute.

-- Starting concurrent execution on Mon Oct 10 19:45:52 2016 with 469.198 GB free disk space (1 processes; 1 concurrently)

/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/1-overlapper/overlap.sh 1 > /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/1-overlapper/overlap.000001.out 2>&1

-- Finished on Mon Oct 10 19:53:01 2016 (429 seconds) with 469.115 GB free disk space

-- Found 1 overlapInCore output files.

-- overlapInCore compute '/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/1-overlapper': -- kmer hits -- with no overlap 10820127 10820127 +- 0

-- with an overlap 5599360 5599360 +- 0

-- overlaps 5599360 5599360 +- 0 -- contained 4278090 4278090 +- 0

-- dovetail 1321270 1321270 +- 0

-- overlaps rejected -- multiple per pair 0 0 +- 0 -- bad short window 0 0 +- 0

-- bad long window 0 0 +- 0

-- Starting command on Mon Oct 10 19:53:01 2016 with 469.115 GB free disk space

/home/freddy/bin/canu/Linux-amd64/bin/ovStoreBuild \
 -O /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/9332_mogene_B01.ovlStore.BUILDING \
 -G /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/9332_mogene_B01.gkpStore \
 -M 2-8 \
 -L /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/1-overlapper/ovljob.files \
 > /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/9332_mogene_B01.ovlStore.err 2>&1

-- Finished on Mon Oct 10 19:53:07 2016 (6 seconds) with 468.822 GB free disk space

-- Overlap store '/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/9332_mogene_B01.ovlStore' successfully constructed.

-- Purged 0.08 GB in 3 overlap output files and 1 directories.

-- Starting command on Mon Oct 10 19:53:07 2016 with 468.903 GB free disk space

/home/freddy/bin/canu/Linux-amd64/bin/ovStoreStats \
 -G /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/9332_mogene_B01.gkpStore \
 -O /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/9332_mogene_B01.ovlStore \
 -o /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/9332_mogene_B01.ovlStore \
 > /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/9332_mogene_B01.ovlStore.summary.err 2>&1

-- Finished on Mon Oct 10 19:53:09 2016 (2 seconds) with 468.902 GB free disk space

-- Overlap store '/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/9332_mogene_B01.ovlStore' contains:

-- category reads % read length feature size or coverage analysis

-- middle-missing 449 2.78 3179.06 +- 1229.90 427.90 +- 426.87 (bad trimming) -- middle-hump 56 0.35 2804.93 +- 1178.86 566.66 +- 671.54 (bad trimming) -- no-5-prime 285 1.76 2398.01 +- 797.66 370.80 +- 483.27 (bad trimming)

-- no-3-prime 293 1.81 2240.24 +- 815.30 242.33 +- 369.45 (bad trimming)

-- low-coverage 1067 6.61 2141.30 +- 941.69 4.29 +- 1.93 (easy to assemble, potential for lower quality consensus) -- unique 2893 17.92 2044.14 +- 742.22 21.63 +- 8.89 (easy to assemble, perfect, yay) -- repeat-cont 5176 32.05 2333.75 +- 757.25 1447.01 +- 1290.48 (potential for consensus errors, no impact on assembly)

-- repeat-dove 11 0.07 3532.36 +- 842.29 716.70 +- 1014.36 (hard to assemble, likely won't assemble correctly or even at all)

-- span-repeat 941 5.83 2698.68 +- 1157.89 1437.14 +- 1217.33 (read spans a large repeat, usually easy to assemble) -- uniq-repeat-cont 3861 23.91 2242.83 +- 690.59 (should be uniquely placed, low potential for consensus errors, no impact on assembly) -- uniq-repeat-dove 796 4.93 3001.63 +- 645.58 (will end contigs, potential to misassemble) -- uniq-anchor 115 0.71 3846.08 +- 991.92 591.47 +- 849.43 (repeat read, with unique section, probable bad read)

Configure RED for 2gb memory with batches of at most 5000 reads and (unlimited) bases.

RED job 1 from read 1 to read 5000 - 0.576 GB for 5000 reads - 0.143 GB for 11795454 olaps - 0.038 GB for evidence RED job 2 from read 5001 to read 10000 - 0.571 GB for 5000 reads - 0.141 GB for 11642554 olaps - 0.039 GB for evidence RED job 3 from read 10001 to read 15000 - 0.563 GB for 5000 reads - 0.139 GB for 11495875 olaps - 0.039 GB for evidence RED job 4 from read 15001 to read 16148 - 0.135 GB for 1148 reads - 0.033 GB for 2740694 olaps - 0.009 GB for evidence

-- read error detection attempt 1 begins with 0 finished, and 4 to compute.

-- Starting concurrent execution on Mon Oct 10 19:53:10 2016 with 468.902 GB free disk space (4 processes; 2 concurrently)

/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/3-overlapErrorAdjustment/red.sh 1 > /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/3-overlapErrorAdjustment/red.000001.out 2>&1
/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/3-overlapErrorAdjustment/red.sh 2 > /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/3-overlapErrorAdjustment/red.000002.out 2>&1
/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/3-overlapErrorAdjustment/red.sh 3 > /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/3-overlapErrorAdjustment/red.000003.out 2>&1
/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/3-overlapErrorAdjustment/red.sh 4 > /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/3-overlapErrorAdjustment/red.000004.out 2>&1

-- Finished on Mon Oct 10 19:54:04 2016 (54 seconds) with 468.901 GB free disk space

-- Found 4 read error detection output files.

Configure OEA for 1gb memory with batches of at most 25000 reads and (unlimited) bases. Configured 1 jobs.

OEA job 1 from read 1 to read 16148 - 35.9 bases + 2.1 adjusts + 0.5 reads + 341.8 olaps + 8.0 fseq/rseq + 32.0 fadj/radj + 32.0 work + 256.0 misc = 708.3 MB

-- overlap error adjustment attempt 1 begins with 0 finished, and 1 to compute.

-- Starting concurrent execution on Mon Oct 10 19:54:05 2016 with 468.901 GB free disk space (1 processes; 8 concurrently)

/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/3-overlapErrorAdjustment/oea.sh 1 > /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/3-overlapErrorAdjustment/oea.000001.out 2>&1

-- Finished on Mon Oct 10 19:55:53 2016 (108 seconds) with 468.879 GB free disk space

-- Found 1 overlap error adjustment output files.

-- Starting command on Mon Oct 10 19:55:53 2016 with 468.879 GB free disk space

/home/freddy/bin/canu/Linux-amd64/bin/ovStoreBuild \
  -G /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/9332_mogene_B01.gkpStore \
  -O /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/9332_mogene_B01.ovlStore \
  -evalues \
  -L /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/3-overlapErrorAdjustment/oea.files \
> /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/3-overlapErrorAdjustment/oea.apply.err 2>&1

-- Finished on Mon Oct 10 19:55:53 2016 (lickety-split) with 468.858 GB free disk space

-- Unitigger attempt 1 begins.

-- Starting concurrent execution on Mon Oct 10 19:55:53 2016 with 468.858 GB free disk space (1 processes; 2 concurrently)

/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/4-unitigger/unitigger.sh 1 > /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/4-unitigger/unitigger.000001.out 2>&1

-- Finished on Mon Oct 10 19:56:00 2016 (7 seconds) with 468.85 GB free disk space

-- Unitigger failed.

-- Unitigger attempt 2 begins.

-- Starting concurrent execution on Mon Oct 10 19:56:00 2016 with 468.85 GB free disk space (1 processes; 2 concurrently)

/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/4-unitigger/unitigger.sh 1 > /home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/4-unitigger/unitigger.000001.out 2>&1

-- Finished on Mon Oct 10 19:56:06 2016 (6 seconds) with 468.85 GB free disk space

-- Unitigger failed.

Don't panic, but a mostly harmless error occurred and canu failed.

canu failed with 'failed to generate unitigs. Made 2 attempts, jobs still failed'.

=> unitigger.err (in 4-unitigger)

Graph error threshold = 0.030 (3.000%) Max error threshold = 0.030 (3.000%)

Minimum overlap length = 500 bases

number of threads = 4 (command line)

==> LOADING AND FILTERING OVERLAPS.

ReadInfo()-- Using 16148 reads, no minimum read length used.

OverlapCache()-- limited to 6144MB memory (user supplied). PHYS_PAGES = 3051251 PAGE_SIZE = 4096 MEMORY = 12497924096 OverlapCache()-- 0MB for read data. OverlapCache()-- 0MB for best edges. OverlapCache()-- 0MB for unitig layouts. OverlapCache()-- 0MB for tigs. OverlapCache()-- 0MB for id maps. OverlapCache()-- 0MB for error profiles. OverlapCache()-- 0MB for overlap cache pointers. OverlapCache()-- 56MB for overlap cache initial bucket. OverlapCache()-- 64MB for overlap cache thread data. OverlapCache()-- 0MB for number of overlaps per read. OverlapCache()-- 0MB for other processes. OverlapCache()-- --------- OverlapCache()-- 121MB for data structures (sum of above). OverlapCache()-- 6022MB available for overlaps.

OverlapCache()-- Loading number of overlaps per read. OverlapCache()-- Initial guess at _maxPer=24443 (max of 4822) from (memLimit=6315325907 - memUsed=0) / (numReads=16148 * sizeof(OVL)=16) OverlapCache()-- _maxPer= 24443 (numBelow=16148 numEqual=0 numAbove=0 totalLoad=11198720 -- 0 + 11198720 = 179179520 <? 6315325907

OverlapCache()-- blockSize = 8388608 (128MB)

OverlapCache()-- _maxPer = 24443 overlaps/reads OverlapCache()-- numBelow = 16148 reads (all overlaps loaded) OverlapCache()-- numEqual = 0 reads (all overlaps loaded) OverlapCache()-- numAbove = 0 reads (some overlaps loaded) OverlapCache()-- totalLoad = 11198720 overlaps (100.00%)

OverlapCache()-- availForOverlaps = 6022MB OverlapCache()-- totalMemory = 0MB for organization OverlapCache()-- totalMemory = 170MB for overlaps OverlapCache()-- totalMemory = 170MB used

OverlapCache()-- Loading: overlaps processed 2641 (000.02%) loaded 2636 (000.02%) droppeddupe 4 (000.00%) (at read iid 1) OverlapCache()-- Loading: overlaps processed 11263856 (100.58%) loaded 10806092 (096.49%) droppeddupe 377126 (003.37%)

BestOverlapGraph()-- allocating best edges (0MB) BestOverlapGraph()-- analyzing 16148 reads for best contains, with 4 threads. BestOverlapGraph()-- analyzing 16148 reads for best edges, with 4 threads. BestOverlapGraph()-- removing suspicious reads from graph, with 4 threads. BestOverlapGraph()-- marked 378 reads as suspicious. BestOverlapGraph()-- analyzing 16148 reads for best contains, with 4 threads. BestOverlapGraph()-- analyzing 16148 reads for best edges, with 4 threads. BestOverlapGraph()-- analyzing best edges to find useful edge error rate BestOverlapGraph()-- analyzing 16148 reads for best contains, with 4 threads. BestOverlapGraph()-- analyzing 16148 reads for best edges, with 4 threads. BestOverlapGraph()-- removing suspicious edges from graph, with 4 threads. BestOverlapGraph()-- analyzing 16148 reads for best contains, with 4 threads. BestOverlapGraph()-- analyzing 16148 reads for best edges, with 4 threads. BestOverlapGraph()-- detecting spur reads. BestOverlapGraph()-- detected 1014 spur reads. BestOverlapGraph()-- analyzing 16148 reads for best contains, with 4 threads. BestOverlapGraph()-- analyzing 16148 reads for best edges, with 4 threads. BestOverlapGraph()-- removing best edges for contained reads.

==> BUILDING GREEDY TIGS.

breakSingletonTigs()-- Removed 998 singleton tigs; reads are now unplaced.

==> PLACE CONTAINED READS.

computeErrorProfiles()-- Computing error profiles for 1180 tigs, with 4 threads.

placeContains()-- placing 13059 contained and 1672 unplaced reads, with 4 threads. placeContains()-- Placed 8465 contained reads and 111 unplaced reads. placeContains()-- Failed to place 4594 contained reads (too high error suspected) and 1561 unplaced reads (lack of overlaps suspected).

==> MERGE ORPHANS.

computeErrorProfiles()-- Computing error profiles for 1180 tigs, with 4 threads.

bubbleDetect()-- working on 1180 tigs, with 4 threads. popBubbles()-- Found 21 potential bubbles. popBubbles()-- tig 468 BUBBLE -> tig 8 popBubbles()-- placed 0 unique orphan tigs popBubbles()-- shattered 0 repeat orphan tigs popBubbles()-- marked 1 unique bubble tigs popBubbles()-- marked 0 repeat bubble tigs

==> GENERATING ASSEMBLY GRAPH.

computeErrorProfiles()-- Computing error profiles for 1180 tigs, with 4 threads.

AssemblyGraph()-- allocating vectors for placements, 0.739MB AssemblyGraph()-- finding edges for 9993 reads (8465 contained), ignoring 6155 unplaced reads, with 4 threads. AssemblyGraph()-- building reverse edges. AssemblyGraph()-- build complete. AssemblyGraph()-- generating '/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/4-unitigger/9332_mogene_B01.initial.edges.gfa'. AssemblyGraph()-- Found 0 edges to unassembled contigs. AssemblyGraph()-- 0 bubble placements AssemblyGraph()-- 0 repeat placements

AssemblyGraph()-- Intratig edges: 0 contained 0 5' 0 3' (in both contig and unitig) AssemblyGraph()-- Contig only edges: 1347 contained 8575 5' 8577 3' AssemblyGraph()-- Unitig only edges: 0 contained 0 5' 0 3' AssemblyGraph()-- Intercontig edges: 5208 contained 13737 5' 13287 3' (in neither contig nor unitig)

==> BREAK REPEATS.

computeErrorProfiles()-- Computing error profiles for 1180 tigs, with 4 threads.

==> CLEANUP MISTAKES.

splitDiscontinuous()-- Tested 195 tigs, split none. promoteToSingleton()-- Moved 6155 unplaced reads to singleton tigs.

==> CLEANUP GRAPH.

AssemblyGraph()-- rebuilding AssemblyGraph()-- building reverse edges. AssemblyGraph()-- rebuild complete. AssemblyGraph()-- filtering edges AssemblyGraph()-- 10995 contig edges and 0 unitig edges. AssemblyGraph()-- 0 bubble edges and 0 repeat edges. AssemblyGraph()-- 26519 middle contig edges filtered from 4086 reads. AssemblyGraph()-- 0 repeat end edges filtered from 0 reads. AssemblyGraph()-- 26519 repeat edges (not output). AssemblyGraph()-- 167 bubble edges. AssemblyGraph()-- 167 intersecting edges (from the end of a tig to somewhere else).

==> GENERATE OUTPUTS.

classifyAsUnassembled()-- 6158 tigs 15210560 bases -- too few reads classifyAsUnassembled()-- 0 tigs 0 bases -- too short classifyAsUnassembled()-- 61 tigs 230095 bases -- single spanning read classifyAsUnassembled()-- 0 tigs 0 bases -- low coverage classifyAsUnassembled()-- 134 tigs 1133423 bases -- acceptable contigs AssemblyGraph()-- generating '/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/4-unitigger/9332_mogene_B01.final.edges.gfa'. AssemblyGraph()-- Found 16669 edges to unassembled contigs. AssemblyGraph()-- 0 bubble placements AssemblyGraph()-- 0 repeat placements

AssemblyGraph()-- Intratig edges: 0 contained 0 5' 0 3' (in both contig and unitig) AssemblyGraph()-- Contig only edges: 1046 contained 6578 5' 6563 3' AssemblyGraph()-- Unitig only edges: 0 contained 0 5' 0 3' AssemblyGraph()-- Intercontig edges: 0 contained 38 5' 59 3' (in neither contig nor unitig)

==> GENERATE UNITIGS.

splitDiscontinuous()-- Tested 163 tigs, split 3 tigs into 6 new tigs. AssemblyGraph()-- generating '/home/freddy/Desktop/mogene_unc/canu/9332/9332_mogene_B01//unitigging/4-unitigger/9332_mogene_B01.unitigs.gfa'.

Failed with 'Segmentation fault'

Backtrace (mangled):

/home/freddy/bin/canu/Linux-amd64/bin/bogart[0x50d590] /lib/x86_64-linux-gnu/libpthread.so.0(+0x113d0)[0x7f48c56503d0] /home/freddy/bin/canu/Linux-amd64/bin/bogart[0x4f5058] /home/freddy/bin/canu/Linux-amd64/bin/bogart[0x4041f3] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f48c5296830] /home/freddy/bin/canu/Linux-amd64/bin/bogart[0x404b49]

Backtrace (demangled):

[0] /home/freddy/bin/canu/Linux-amd64/bin/bogart() [0x50d590] [1] /lib/x86_64-linux-gnu/libpthread.so.0::(null) + 0x113d0 [0x7f48c56503d0] [2] /home/freddy/bin/canu/Linux-amd64/bin/bogart() [0x4f5058] [3] /home/freddy/bin/canu/Linux-amd64/bin/bogart() [0x4041f3] [4] /lib/x86_64-linux-gnu/libc.so.6::(null) + 0xf0 [0x7f48c5296830] [5] /home/freddy/bin/canu/Linux-amd64/bin/bogart() [0x404b49]

GDB:

Segmentation fault

freddymonteiro commented 7 years ago

Installed the 1.3 binary package. Removed 4-unitigger/ Run canu-1.3 on the sample. Successful assembly.

Sorry for the trouble