search

marbl / canu

A single molecule sequence assembler for genomes large and small.
http://canu.readthedocs.io/
657 stars 179 forks source link

Assembly but no corrected reads v1.7 r8692 #915

Closed damientully closed 6 years ago

damientully commented 6 years ago

Hi guys,

I am running Canu v1.7 +0 changes (r8692 c9ef9219a265e0bbe3a311cca7d28aa02b7517d3) and while I get an assembly I do not get any corrected reads. My command is

./canu -useGrid=0 -p BC69 -d /n/scratch2/dct7/Run22/BC69 genomeSize=2000 -nanopore-raw /n/scratch2/dct7/Run22/BC69.fastq minReadLength=1900 corOutCoverage=1000 stopOnReadQuality=false 

Restoring modules from user's modules
-- Canu snapshot v1.7 +184 changes (r8876 098679d3a315131e04658b0351f9031b52a85d7d)
--
-- CITATIONS
--
-- Koren S, Walenz BP, Berlin K, Miller JR, Phillippy AM.
-- Canu: scalable and accurate long-read assembly via adaptive k-mer weighting and repeat separation.
-- Genome Res. 2017 May;27(5):722-736.
-- http://doi.org/10.1101/gr.215087.116
-- 
-- Read and contig alignments during correction, consensus and GFA building use:
--   Šošic M, Šikic M.
--   Edlib: a C/C ++ library for fast, exact sequence alignment using edit distance.
--   Bioinformatics. 2017 May 1;33(9):1394-1395.
--   http://doi.org/10.1093/bioinformatics/btw753
-- 
-- Overlaps are generated using:
--   Berlin K, et al.
--   Assembling large genomes with single-molecule sequencing and locality-sensitive hashing.
--   Nat Biotechnol. 2015 Jun;33(6):623-30.
--   http://doi.org/10.1038/nbt.3238
-- 
--   Myers EW, et al.
--   A Whole-Genome Assembly of Drosophila.
--   Science. 2000 Mar 24;287(5461):2196-204.
--   http://doi.org/10.1126/science.287.5461.2196
-- 
-- Corrected read consensus sequences are generated using an algorithm derived from FALCON-sense:
--   Chin CS, et al.
--   Phased diploid genome assembly with single-molecule real-time sequencing.
--   Nat Methods. 2016 Dec;13(12):1050-1054.
--   http://doi.org/10.1038/nmeth.4035
-- 
-- Contig consensus sequences are generated using an algorithm derived from pbdagcon:
--   Chin CS, et al.
--   Nonhybrid, finished microbial genome assemblies from long-read SMRT sequencing data.
--   Nat Methods. 2013 Jun;10(6):563-9
--   http://doi.org/10.1038/nmeth.2474
-- 
-- CONFIGURE CANU
--
-- Detected Java(TM) Runtime Environment '1.8.0_112' (from '/n/app/java/jdk-1.8u112/bin/java').
-- Detected gnuplot version '4.6 patchlevel 2' (from 'gnuplot') and image format 'png'.
-- Detected 32 CPUs and 252 gigabytes of memory.
-- Detected Slurm with 'sinfo' binary in /usr/bin/sinfo.
-- Grid engine disabled per useGrid=false option.
--
--                            (tag)Concurrency
--                     (tag)Threads          |
--            (tag)Memory         |          |
--        (tag)         |         |          |     total usage     algorithm
--        -------  ------  --------   --------  -----------------  -----------------------------
-- Local: meryl      8 GB    4 CPUs x   1 job      8 GB    4 CPUs  (k-mer counting)
-- Local: cormhap    6 GB   16 CPUs x   2 jobs    12 GB   32 CPUs  (overlap detection with mhap)
-- Local: obtovl     4 GB    8 CPUs x   4 jobs    16 GB   32 CPUs  (overlap detection)
-- Local: utgovl     4 GB    8 CPUs x   4 jobs    16 GB   32 CPUs  (overlap detection)
-- Local: ovb        4 GB    1 CPU  x  32 jobs   128 GB   32 CPUs  (overlap store bucketizer)
-- Local: ovs        8 GB    1 CPU  x  31 jobs   248 GB   31 CPUs  (overlap store sorting)
-- Local: red        8 GB    4 CPUs x   8 jobs    64 GB   32 CPUs  (read error detection)
-- Local: oea        4 GB    1 CPU  x  32 jobs   128 GB   32 CPUs  (overlap error adjustment)
-- Local: bat       16 GB    4 CPUs x   1 job     16 GB    4 CPUs  (contig construction with bogart)
-- Local: gfa        8 GB    4 CPUs x   1 job      8 GB    4 CPUs  (GFA alignment and processing)
--
-- Found Nanopore uncorrected reads in the input files.
--
-- Generating assembly 'BC69' in '/n/scratch2/dct7/Run22/BC69'
--
-- Parameters:
--
--  genomeSize        2000
--
--  Overlap Generation Limits:
--    corOvlErrorRate 0.3200 ( 32.00%)
--    obtOvlErrorRate 0.1440 ( 14.40%)
--    utgOvlErrorRate 0.1440 ( 14.40%)
--
--  Overlap Processing Limits:
--    corErrorRate    0.5000 ( 50.00%)
--    obtErrorRate    0.1440 ( 14.40%)
--    utgErrorRate    0.1440 ( 14.40%)
--    cnsErrorRate    0.1920 ( 19.20%)
--
--
-- BEGIN CORRECTION
--
----------------------------------------
-- Starting command on Sun May 13 05:02:03 2018 with 510807.942 GB free disk space

    cd .
    /home/dct7/canu/Linux-amd64/bin/sqStoreCreate \
      -o ./BC69.seqStore.BUILDING \
      -minlength 1850 \
      ./BC69.seqStore.ssi \
    > ./BC69.seqStore.BUILDING.err 2>&1

-- Finished on Sun May 13 05:02:04 2018 (one second) with 510807.942 GB free disk space
----------------------------------------
--
-- In sequence store './BC69.seqStore':
--   Found 34602 reads.
--   Found 65832795 bases (32916.39 times coverage).
--
--   Read length histogram (one '*' equals 250.07 reads):
--     1800   1899  17505 **********************************************************************
--     1900   1999  16755 *******************************************************************
--     2000   2099    266 *
--     2100   2199     19 
--     2200   2299      3 
--     2300   2399      7 
--     2400   2499      4 
--     2500   2599      3 
--     2600   2699      5 
--     2700   2799      7 
--     2800   2899      5 
--     2900   2999      1 
--     3000   3099      3 
--     3100   3199      3 
--     3200   3299      2 
--     3300   3399      1 
--     3400   3499      3 
--     3500   3599      7 
--     3600   3699      3 
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'meryl' concurrent execution on Sun May 13 05:02:05 2018 with 510808.781 GB free disk space (1 processes; 1 concurrently)

    cd correction/0-mercounts
    ./meryl.sh 1 > ./meryl.000001.out 2>&1

-- Finished on Sun May 13 05:02:11 2018 (6 seconds) with 510819.599 GB free disk space
----------------------------------------
-- Meryl finished successfully.
--
--  16-mers                                                                                           Fraction
--    Occurrences   NumMers                                                                         Unique Total
--       1-     1   8434721 *******************************************************************--> 0.7457 0.1291
--       2-     2   1179526 ********************************************************************** 0.8500 0.1653
--       3-     4    731203 *******************************************                            0.8918 0.1870
--       5-     7    363272 *********************                                                  0.9292 0.2155
--       8-    11    193120 ***********                                                            0.9525 0.2428
--      12-    16    113168 ******                                                                 0.9664 0.2677
--      17-    22     72011 ****                                                                   0.9752 0.2900
--      23-    29     48521 **                                                                     0.9810 0.3103
--      30-    37     34383 **                                                                     0.9849 0.3287
--      38-    46     25049 *                                                                      0.9878 0.3458
--      47-    56     18918 *                                                                      0.9899 0.3614
--      57-    67     14882                                                                        0.9915 0.3760
--      68-    79     11626                                                                        0.9928 0.3898
--      80-    92      9664                                                                        0.9938 0.4028
--      93-   106      7870                                                                        0.9947 0.4153
--     107-   121      6443                                                                        0.9953 0.4272
--     122-   137      5505                                                                        0.9959 0.4382
--     138-   154      4753                                                                        0.9964 0.4492
--     155-   172      3895                                                                        0.9968 0.4596
--     173-   191      3155                                                                        0.9971 0.4693
--     192-   211      2900                                                                        0.9974 0.4780
--     212-   232      2507                                                                        0.9977 0.4869
--     233-   254      2067                                                                        0.9979 0.4954
--     255-   277      1865                                                                        0.9981 0.5029
--     278-   301      1739                                                                        0.9982 0.5106
--     302-   326      1490                                                                        0.9984 0.5183
--     327-   352      1323                                                                        0.9985 0.5254
--     353-   379      1151                                                                        0.9986 0.5323
--     380-   407      1133                                                                        0.9987 0.5387
--     408-   436       902                                                                        0.9988 0.5455
--     437-   466       835                                                                        0.9989 0.5513
--     467-   497       806                                                                        0.9990 0.5570
--     498-   529       683                                                                        0.9991 0.5629
--     530-   562       597                                                                        0.9991 0.5683
--     563-   596       578                                                                        0.9992 0.5733
--     597-   631       562                                                                        0.9992 0.5784
--     632-   667       536                                                                        0.9993 0.5837
--     668-   704       434                                                                        0.9993 0.5890
--     705-   742       431                                                                        0.9994 0.5936
--     743-   781       421                                                                        0.9994 0.5983
--     782-   821       339                                                                        0.9994 0.6032
--
--       30146 (max occurrences)
--    56879044 (total mers, non-unique)
--     2876324 (distinct mers, non-unique)
--     8434721 (unique mers)
-- For mhap overlapping, the threshold 0.000005000000000 is too low, resetting to 0.001259918181106 to capture 5.000000 * 32916 coverage.
-- For mhap overlapping, set repeat k-mer threshold to 164580.
--
-- Found 65313765 16-mers; 11311045 distinct and 8434721 unique.  Largest count 30146.
--
-- OVERLAPPER (mhap) (correction)
--
-- Set corMhapSensitivity=low based on read coverage of 32916.
--
-- PARAMETERS: hashes=256, minMatches=3, threshold=0.85
--
-- Given 6 GB, can fit 18000 reads per block.
-- For 3 blocks, set stride to 2 blocks.
-- Logging partitioning to 'correction/1-overlapper/partitioning.log'.
-- Configured 2 mhap precompute jobs.
-- Configured 2 mhap overlap jobs.
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'cormhap' concurrent execution on Sun May 13 05:02:11 2018 with 510819.599 GB free disk space (2 processes; 2 concurrently)

    cd correction/1-overlapper
    ./precompute.sh 1 > ./precompute.000001.out 2>&1
    ./precompute.sh 2 > ./precompute.000002.out 2>&1

-- Finished on Sun May 13 05:03:02 2018 (51 seconds) with 511021.285 GB free disk space
----------------------------------------
-- All 2 mhap precompute jobs finished successfully.
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'cormhap' concurrent execution on Sun May 13 05:03:02 2018 with 511021.285 GB free disk space (2 processes; 2 concurrently)

    cd correction/1-overlapper
    ./mhap.sh 1 > ./mhap.000001.out 2>&1
    ./mhap.sh 2 > ./mhap.000002.out 2>&1

-- Finished on Sun May 13 06:08:01 2018 (3899 seconds) with 524235.989 GB free disk space
----------------------------------------
-- Found 2 mhap overlap output files.
----------------------------------------
-- Starting command on Sun May 13 06:08:01 2018 with 524235.989 GB free disk space

    cd correction
    /home/dct7/canu/Linux-amd64/bin/ovStoreConfig \
     -S ../BC69.seqStore \
     -M 4-8 \
     -L ./1-overlapper/ovljob.files \
     -create ./BC69.ovlStore.config \
     > ./BC69.ovlStore.config.txt \
    2> ./BC69.ovlStore.config.err

-- Finished on Sun May 13 06:08:01 2018 (fast as lightning) with 524235.989 GB free disk space
----------------------------------------
--
-- Creating overlap store correction/BC69.ovlStore using:
--      2 buckets
--      3 slices
--        using at most 7 GB memory each
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'ovB' concurrent execution on Sun May 13 06:08:01 2018 with 524235.989 GB free disk space (2 processes; 32 concurrently)

    cd correction/BC69.ovlStore.BUILDING
    ./scripts/1-bucketize.sh 1 > ./logs/1-bucketize.000001.out 2>&1
    ./scripts/1-bucketize.sh 2 > ./logs/1-bucketize.000002.out 2>&1

-- Finished on Sun May 13 06:11:16 2018 (195 seconds) with 525141.681 GB free disk space
----------------------------------------
-- Overlap store bucketizer finished.
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'ovS' concurrent execution on Sun May 13 06:11:16 2018 with 525141.681 GB free disk space (3 processes; 31 concurrently)

    cd correction/BC69.ovlStore.BUILDING
    ./scripts/2-sort.sh 1 > ./logs/2-sort.000001.out 2>&1
    ./scripts/2-sort.sh 2 > ./logs/2-sort.000002.out 2>&1
    ./scripts/2-sort.sh 3 > ./logs/2-sort.000003.out 2>&1

-- Finished on Sun May 13 06:13:36 2018 (140 seconds) with 525195.992 GB free disk space
----------------------------------------
-- Overlap store sorter finished.
----------------------------------------
-- Starting command on Sun May 13 06:13:36 2018 with 525195.992 GB free disk space

    cd correction
    /home/dct7/canu/Linux-amd64/bin/ovStoreIndexer \
      -O  ./BC69.ovlStore.BUILDING \
      -S ../BC69.seqStore \
      -C  ./BC69.ovlStore.config \
      -delete \
    > ./BC69.ovlStore.BUILDING.index.err 2>&1

-- Finished on Sun May 13 06:13:36 2018 (like a bat out of hell) with 525195.992 GB free disk space
----------------------------------------
FROM: 'correction/BC69.ovlStore.BUILDING'
TO:   'correction/BC69.ovlStore'
-- Checking store.
----------------------------------------
-- Starting command on Sun May 13 06:13:36 2018 with 525195.992 GB free disk space

    cd correction
    /home/dct7/canu/Linux-amd64/bin/ovStoreDump \
     -S ../BC69.seqStore \
     -O  ./BC69.ovlStore \
     -counts \
     > ./BC69.ovlStore/counts.dat 2> ./BC69.ovlStore/counts.err

-- Finished on Sun May 13 06:13:36 2018 (in the blink of an eye) with 525195.992 GB free disk space
----------------------------------------
--
-- Overlap store 'correction/BC69.ovlStore' successfully constructed.
-- Found 834084218 overlaps for 34596 reads; 6 reads have no overlaps.
--
--
-- Purged 6.626 GB in 7 overlap output files.
-- Set corMinCoverage=4 based on read coverage of 32916.
-- Global filter scores will be estimated.
-- Computing correction layouts.
----------------------------------------
-- Starting command on Sun May 13 06:13:36 2018 with 525195.992 GB free disk space

    cd correction
    /home/dct7/canu/Linux-amd64/bin/generateCorrectionLayouts \
      -S ../BC69.seqStore \
      -O  ./BC69.ovlStore \
      -C  ./BC69.corStore.WORKING \
      -eC 80 \
    > ./BC69.corStore.err 2>&1

-- Finished on Sun May 13 06:14:31 2018 (55 seconds) with 525225.211 GB free disk space
----------------------------------------
-- Computing correction layouts.
----------------------------------------
-- Starting command on Sun May 13 06:14:31 2018 with 525225.211 GB free disk space

    cd correction/2-correction
    /home/dct7/canu/Linux-amd64/bin/filterCorrectionLayouts \
      -S  ../../BC69.seqStore \
      -C     ../BC69.corStore \
      -R      ./BC69.readsToCorrect.WORKING \
      -cc 4 \
      -cl 1850 \
      -g  2000 \
      -c  40 \
    > ./BC69.readsToCorrect.err 2>&1

-- Finished on Sun May 13 06:14:31 2018 (lickety-split) with 525225.211 GB free disk space
----------------------------------------
--                             original      original
--                            raw reads     raw reads
--   category                w/overlaps  w/o/overlaps
--   -------------------- ------------- -------------
--   Number of Reads              21196         13406
--   Number of Bases           40436009      25384888
--   Coverage                 20218.004     12692.444
--   Median                        1905          1889
--   Mean                          1907          1893
--   N50                           1905          1890
--   Minimum                       1850             0
--   Maximum                       3628          2498
--   
--                                        --------corrected---------  ----------rescued----------
--                             evidence                     expected                     expected
--   category                     reads            raw     corrected            raw     corrected
--   -------------------- -------------  ------------- -------------  ------------- -------------
--   Number of Reads                291             28            28          10754         10754
--   Number of Bases             600908          85581         81660       20572347      20392117
--   Coverage                   300.454         42.791        40.830      10286.174     10196.058
--   Median                        1900           3134          3055           1912          1896
--   Mean                          2064           3056          2916           1912          1896
--   N50                           1904           3224          3209           1913          1896
--   Minimum                       1850           2174          2161           1855          1851
--   Maximum                       3628           3613          3566           2163          2150
--   
--                        --------uncorrected--------
--                                           expected
--   category                       raw     corrected
--   -------------------- ------------- -------------
--   Number of Reads              23820         23820
--   Number of Bases           45162969       2446213
--   Coverage                 22581.484      1223.107
--   Median                        1892             0
--   Mean                          1896           102
--   N50                           1892          1833
--   Minimum                          0             0
--   Maximum                       3628          3606
--   
--   Maximum Memory           557909996
--
--                            (tag)Concurrency
--                     (tag)Threads          |
--            (tag)Memory         |          |
--        (tag)         |         |          |     total usage     algorithm
--        -------  ------  --------   --------  -----------------  -----------------------------
-- Local: cor        2 GB    4 CPUs x   8 jobs    16 GB   32 CPUs  (read correction)
--
-- Using overlaps no worse than 0.5 fraction error for correcting reads (from corErrorRate parameter).
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'cor' concurrent execution on Sun May 13 06:14:31 2018 with 525225.211 GB free disk space (7 processes; 8 concurrently)

    cd correction/2-correction
    ./correctReads.sh 1 > ./correctReads.000001.out 2>&1
    ./correctReads.sh 2 > ./correctReads.000002.out 2>&1
    ./correctReads.sh 3 > ./correctReads.000003.out 2>&1
    ./correctReads.sh 4 > ./correctReads.000004.out 2>&1
    ./correctReads.sh 5 > ./correctReads.000005.out 2>&1
    ./correctReads.sh 6 > ./correctReads.000006.out 2>&1
    ./correctReads.sh 7 > ./correctReads.000007.out 2>&1

-- Finished on Sun May 13 06:16:19 2018 (108 seconds) with 525318.362 GB free disk space
----------------------------------------
-- Found 7 read correction output files.
-- Found 7 read correction output files.
--
-- Loading corrected reads into corStore and seqStore.
----------------------------------------
-- Starting command on Sun May 13 06:16:19 2018 with 525318.362 GB free disk space

    cd correction
    /home/dct7/canu/Linux-amd64/bin/loadCorrectedReads \
      -S ../BC69.seqStore \
      -C ./BC69.corStore \
      -L ./2-correction/corjob.files \
    >  ./BC69.loadCorrectedReads.log \
    2> ./BC69.loadCorrectedReads.err

-- Finished on Sun May 13 06:16:20 2018 (one second) with 525318.362 GB free disk space
----------------------------------------
--
-- In sequence store './BC69.seqStore':
--   Found 10780 reads.
--   Found 20297568 bases (10148.78 times coverage).
--
--   Read length histogram (one '*' equals 117.65 reads):
--     1300   1399      1 
--     1400   1499      0 
--     1500   1599      0 
--     1600   1699      6 
--     1700   1799     55 
--     1800   1899   8236 **********************************************************************
--     1900   1999   2462 ********************
--     2000   2099      1 
--     2100   2199      1 
--     2200   2299      2 
--     2300   2399      0 
--     2400   2499      0 
--     2500   2599      0 
--     2600   2699      1 
--     2700   2799      1 
--     2800   2899      0 
--     2900   2999      2 
--     3000   3099      1 
--     3100   3199      1 
--     3200   3299      0 
--     3300   3399      1 
--     3400   3499      5 
--     3500   3599      4 
--
-- Purging correctReads output after loading into stores.
-- Purged 7 .cns outputs.
-- Purged 14 .out job log outputs.
--
-- Purging overlaps used for correction.
--
--
-- BEGIN TRIMMING
--
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'meryl' concurrent execution on Sun May 13 06:16:20 2018 with 525318.362 GB free disk space (1 processes; 1 concurrently)

    cd trimming/0-mercounts
    ./meryl.sh 1 > ./meryl.000001.out 2>&1

-- Finished on Sun May 13 06:16:24 2018 (4 seconds) with 525339.64 GB free disk space
----------------------------------------
-- Meryl finished successfully.
--
--  22-mers                                                                                           Fraction
--    Occurrences   NumMers                                                                         Unique Total
--       1-     1    626607 *******************************************************************--> 0.6893 0.0312
--       2-     2     98393 ********************************************************************** 0.7975 0.0410
--       3-     4     67463 ***********************************************                        0.8447 0.0474
--       5-     7     36402 *************************                                              0.8890 0.0562
--       8-    11     22097 ***************                                                        0.9198 0.0658
--      12-    16     13532 *********                                                              0.9398 0.0751
--      17-    22      9480 ******                                                                 0.9530 0.0839
--      23-    29      6658 ****                                                                   0.9627 0.0929
--      30-    37      4600 ***                                                                    0.9695 0.1010
--      38-    46      3395 **                                                                     0.9742 0.1083
--      47-    56      2613 *                                                                      0.9778 0.1154
--      57-    67      2275 *                                                                      0.9806 0.1220
--      68-    79      1699 *                                                                      0.9830 0.1288
--      80-    92      1447 *                                                                      0.9848 0.1349
--      93-   106      1248                                                                        0.9864 0.1410
--     107-   121      1008                                                                        0.9878 0.1472
--     122-   137       896                                                                        0.9889 0.1529
--     138-   154       712                                                                        0.9898 0.1586
--     155-   172       637                                                                        0.9906 0.1638
--     173-   191       499                                                                        0.9913 0.1689
--     192-   211       503                                                                        0.9918 0.1734
--     212-   232       441                                                                        0.9924 0.1784
--     233-   254       344                                                                        0.9929 0.1833
--     255-   277       305                                                                        0.9932 0.1874
--     278-   301       302                                                                        0.9936 0.1913
--     302-   326       290                                                                        0.9939 0.1958
--     327-   352       272                                                                        0.9942 0.2003
--     353-   379       228                                                                        0.9945 0.2049
--     380-   407       167                                                                        0.9948 0.2090
--     408-   436       187                                                                        0.9950 0.2123
--     437-   466       182                                                                        0.9952 0.2162
--     467-   497       182                                                                        0.9954 0.2202
--     498-   529       166                                                                        0.9956 0.2247
--     530-   562       151                                                                        0.9957 0.2289
--     563-   596       144                                                                        0.9959 0.2330
--     597-   631       108                                                                        0.9961 0.2371
--     632-   667       104                                                                        0.9962 0.2405
--     668-   704        78                                                                        0.9963 0.2438
--     705-   742       114                                                                        0.9964 0.2465
--     743-   781        76                                                                        0.9965 0.2506
--     782-   821        78                                                                        0.9966 0.2534
--
--       15187 (max occurrences)
--    19444581 (total mers, non-unique)
--      282495 (distinct mers, non-unique)
--      626607 (unique mers)
----------------------------------------
-- Starting command on Sun May 13 06:16:24 2018 with 525339.64 GB free disk space

    cd trimming/0-mercounts
    /home/dct7/canu/Linux-amd64/bin/meryl \
      -Dt \
      -n 9797 \
      -s ./BC69.ms22 \
    > ./BC69.ms22.frequentMers.fasta \
    2> ./BC69.ms22.frequentMers.fasta.err

-- Finished on Sun May 13 06:16:24 2018 (in the blink of an eye) with 525339.64 GB free disk space
----------------------------------------
-- Reset obtOvlMerThreshold from auto to 9797.
--
-- Found 20071188 22-mers; 909102 distinct and 626607 unique.  Largest count 15187.
--
-- OVERLAPPER (normal) (trimming) erate=0.144
--
----------------------------------------
-- Starting command on Sun May 13 06:16:24 2018 with 525339.64 GB free disk space

    cd trimming/1-overlapper
    /home/dct7/canu/Linux-amd64/bin/overlapInCorePartition \
     -S  ../../BC69.seqStore \
     -hl 85312000 \
     -rl 1000000000 \
     -ol 500 \
     -o  ./BC69.partition \
    > ./BC69.partition.err 2>&1

-- Finished on Sun May 13 06:16:24 2018 (fuirously fast) with 525339.64 GB free disk space
----------------------------------------
--
-- Configured 1 overlapInCore jobs.
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'obtovl' concurrent execution on Sun May 13 06:16:24 2018 with 525339.64 GB free disk space (1 processes; 4 concurrently)

    cd trimming/1-overlapper
    ./overlap.sh 1 > ./overlap.000001.out 2>&1

-- Finished on Sun May 13 09:10:47 2018 (10463 seconds) with 549696.048 GB free disk space
----------------------------------------
-- Found 1 overlapInCore output files.
--
-- overlapInCore compute 'trimming/1-overlapper':
--   kmer hits
--     with no overlap         45273609    45273609 +- 0
--     with an overlap         58113934    58113934 +- 0
--
--   overlaps                  58105483    58105483 +- 0
--     contained                      0           0 +- 0
--     dovetail                       0           0 +- 0
--
--   overlaps rejected
--     multiple per pair              0           0 +- 0
--     bad short window               0           0 +- 0
--     bad long window                0           0 +- 0
----------------------------------------
-- Starting command on Sun May 13 09:10:47 2018 with 549696.048 GB free disk space

    cd trimming
    /home/dct7/canu/Linux-amd64/bin/ovStoreConfig \
     -S ../BC69.seqStore \
     -M 7 \
     -L ./1-overlapper/ovljob.files \
     -create ./BC69.ovlStore.config \
     > ./BC69.ovlStore.config.txt \
    2> ./BC69.ovlStore.config.err

-- Finished on Sun May 13 09:10:48 2018 (one second) with 549696.048 GB free disk space
----------------------------------------
--
-- Creating overlap store trimming/BC69.ovlStore using:
--      1 bucket
--      1 slice
--        using at most 4 GB memory each
----------------------------------------
-- Starting command on Sun May 13 09:10:48 2018 with 549696.048 GB free disk space

    cd trimming
    /home/dct7/canu/Linux-amd64/bin/ovStoreBuild \
     -O  ./BC69.ovlStore.BUILDING \
     -S ../BC69.seqStore \
     -C  ./BC69.ovlStore.config \
     > ./BC69.ovlStore.err 2>&1

-- Finished on Sun May 13 09:11:30 2018 (42 seconds) with 549693.377 GB free disk space
----------------------------------------
-- Checking store.
----------------------------------------
-- Starting command on Sun May 13 09:11:30 2018 with 549693.377 GB free disk space

    cd trimming
    /home/dct7/canu/Linux-amd64/bin/ovStoreDump \
     -S ../BC69.seqStore \
     -O  ./BC69.ovlStore \
     -counts \
     > ./BC69.ovlStore/counts.dat 2> ./BC69.ovlStore/counts.err

-- Finished on Sun May 13 09:11:30 2018 (lickety-split) with 549693.377 GB free disk space
----------------------------------------
--
-- Overlap store 'trimming/BC69.ovlStore' successfully constructed.
-- Found 116227868 overlaps for 10780 reads; 23822 reads have no overlaps.
--
--
-- Purged 0.846 GB in 3 overlap output files.
----------------------------------------
-- Starting command on Sun May 13 09:11:30 2018 with 549693.377 GB free disk space

    cd trimming/3-overlapbasedtrimming
    /home/dct7/canu/Linux-amd64/bin/trimReads \
      -S  ../../BC69.seqStore \
      -O  ../BC69.ovlStore \
      -Co ./BC69.1.trimReads.clear \
      -e  0.144 \
      -minlength 1850 \
      -ol 1 \
      -oc 1 \
      -o  ./BC69.1.trimReads \
    >     ./BC69.1.trimReads.err 2>&1

-- Finished on Sun May 13 09:11:58 2018 (28 seconds) with 549694.015 GB free disk space
----------------------------------------
--  PARAMETERS:
--  ----------
--     1850    (reads trimmed below this many bases are deleted)
--   0.1440    (use overlaps at or below this fraction error)
--        1    (break region if overlap is less than this long, for 'largest covered' algorithm)
--        1    (break region if overlap coverage is less than this many read, for 'largest covered' algorithm)
--  
--  INPUT READS:
--  -----------
--   34602 reads     20297568 bases (reads processed)
--       0 reads            0 bases (reads not processed, previously deleted)
--       0 reads            0 bases (reads not processed, in a library where trimming isn't allowed)
--  
--  OUTPUT READS:
--  ------------
--     958 reads      1803207 bases (trimmed reads output)
--    8535 reads     16119611 bases (reads with no change, kept as is)
--   23822 reads            0 bases (reads with no overlaps, deleted)
--    1287 reads      2367971 bases (reads with short trimmed length, deleted)
--  
--  TRIMMING DETAILS:
--  ----------------
--     433 reads         3223 bases (bases trimmed from the 5' end of a read)
--     556 reads         3556 bases (bases trimmed from the 3' end of a read)
----------------------------------------
-- Starting command on Sun May 13 09:11:58 2018 with 549694.015 GB free disk space

    cd trimming/3-overlapbasedtrimming
    /home/dct7/canu/Linux-amd64/bin/splitReads \
      -S  ../../BC69.seqStore \
      -O  ../BC69.ovlStore \
      -Ci ./BC69.1.trimReads.clear \
      -Co ./BC69.2.splitReads.clear \
      -e  0.144 \
      -minlength 1850 \
      -o  ./BC69.2.splitReads \
    >     ./BC69.2.splitReads.err 2>&1

-- Finished on Sun May 13 09:12:38 2018 (40 seconds) with 549693.817 GB free disk space
----------------------------------------
--  PARAMETERS:
--  ----------
--     1850    (reads trimmed below this many bases are deleted)
--   0.1440    (use overlaps at or below this fraction error)
--  INPUT READS:
--  -----------
--    9493 reads     17929597 bases (reads processed)
--   25109 reads      2367971 bases (reads not processed, previously deleted)
--       0 reads            0 bases (reads not processed, in a library where trimming isn't allowed)
--  
--  PROCESSED:
--  --------
--       0 reads            0 bases (no overlaps)
--       0 reads            0 bases (no coverage after adjusting for trimming done already)
--       0 reads            0 bases (processed for chimera)
--       0 reads            0 bases (processed for spur)
--    9493 reads     17929597 bases (processed for subreads)
--  
--  READS WITH SIGNALS:
--  ------------------
--       0 reads            0 signals (number of 5' spur signal)
--       0 reads            0 signals (number of 3' spur signal)
--       0 reads            0 signals (number of chimera signal)
--       0 reads            0 signals (number of subread signal)
--  
--  SIGNALS:
--  -------
--       0 reads            0 bases (size of 5' spur signal)
--       0 reads            0 bases (size of 3' spur signal)
--       0 reads            0 bases (size of chimera signal)
--       0 reads            0 bases (size of subread signal)
--  
--  TRIMMING:
--  --------
--       0 reads            0 bases (trimmed from the 5' end of the read)
--       0 reads            0 bases (trimmed from the 3' end of the read)
----------------------------------------
-- Starting command on Sun May 13 09:12:38 2018 with 549693.817 GB free disk space

    cd trimming/3-overlapbasedtrimming
    /home/dct7/canu/Linux-amd64/bin/loadTrimmedReads \
      -S ../../BC69.seqStore \
      -c ./BC69.2.splitReads.clear \
    > ./BC69.loadtrimmedReads.err 2>&1

-- Finished on Sun May 13 09:12:38 2018 (fuirously fast) with 549693.817 GB free disk space
----------------------------------------
--
-- In sequence store './BC69.seqStore':
--   Found 9493 reads.
--   Found 17922818 bases (8961.4 times coverage).
--
--   Read length histogram (one '*' equals 101.17 reads):
--     1800   1899   7082 **********************************************************************
--     1900   1999   2397 ***********************
--     2000   2099      0 
--     2100   2199      1 
--     2200   2299      0 
--     2300   2399      0 
--     2400   2499      0 
--     2500   2599      0 
--     2600   2699      1 
--     2700   2799      0 
--     2800   2899      1 
--     2900   2999      0 
--     3000   3099      0 
--     3100   3199      1 
--     3200   3299      1 
--     3300   3399      0 
--     3400   3499      5 
--     3500   3599      4 
--
-- Overlaps used for trimming saved.
--
--
-- BEGIN ASSEMBLY
--
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'meryl' concurrent execution on Sun May 13 09:12:38 2018 with 549693.817 GB free disk space (1 processes; 1 concurrently)

    cd unitigging/0-mercounts
    ./meryl.sh 1 > ./meryl.000001.out 2>&1

-- Finished on Sun May 13 09:12:40 2018 (2 seconds) with 549693.778 GB free disk space
----------------------------------------
-- Meryl finished successfully.
--
--  22-mers                                                                                           Fraction
--    Occurrences   NumMers                                                                         Unique Total
--       1-     1    445313 *******************************************************************--> 0.6657 0.0251
--       2-     2     74895 ********************************************************************** 0.7776 0.0336
--       3-     4     52335 ************************************************                       0.8273 0.0392
--       5-     7     30152 ****************************                                           0.8753 0.0472
--       8-    11     17826 ****************                                                       0.9095 0.0560
--      12-    16     11048 **********                                                             0.9318 0.0647
--      17-    22      7837 *******                                                                0.9466 0.0729
--      23-    29      5261 ****                                                                   0.9571 0.0810
--      30-    37      3793 ***                                                                    0.9645 0.0885
--      38-    46      2791 **                                                                     0.9698 0.0953
--      47-    56      2293 **                                                                     0.9740 0.1020
--      57-    67      1742 *                                                                      0.9772 0.1084
--      68-    79      1438 *                                                                      0.9797 0.1144
--      80-    92      1264 *                                                                      0.9819 0.1203
--      93-   106      1068                                                                        0.9837 0.1265
--     107-   121       902                                                                        0.9853 0.1323
--     122-   137       685                                                                        0.9866 0.1380
--     138-   154       644                                                                        0.9876 0.1430
--     155-   172       547                                                                        0.9886 0.1482
--     173-   191       449                                                                        0.9894 0.1533
--     192-   211       404                                                                        0.9901 0.1581
--     212-   232       316                                                                        0.9906 0.1624
--     233-   254       334                                                                        0.9911 0.1663
--     255-   277       327                                                                        0.9916 0.1709
--     278-   301       280                                                                        0.9921 0.1758
--     302-   326       231                                                                        0.9925 0.1805
--     327-   352       196                                                                        0.9928 0.1845
--     353-   379       184                                                                        0.9931 0.1882
--     380-   407       203                                                                        0.9934 0.1920
--     408-   436       188                                                                        0.9937 0.1967
--     437-   466       145                                                                        0.9940 0.2010
--     467-   497       139                                                                        0.9942 0.2047
--     498-   529       172                                                                        0.9944 0.2085
--     530-   562       126                                                                        0.9947 0.2135
--     563-   596        65                                                                        0.9949 0.2172
--     597-   631        99                                                                        0.9950 0.2194
--     632-   667       117                                                                        0.9951 0.2228
--     668-   704       109                                                                        0.9953 0.2271
--     705-   742        73                                                                        0.9954 0.2314
--     743-   781        54                                                                        0.9956 0.2343
--     782-   821        78                                                                        0.9956 0.2366
--
--       13905 (max occurrences)
--    17278152 (total mers, non-unique)
--      223656 (distinct mers, non-unique)
--      445313 (unique mers)
----------------------------------------
-- Starting command on Sun May 13 09:12:40 2018 with 549693.778 GB free disk space

    cd unitigging/0-mercounts
    /home/dct7/canu/Linux-amd64/bin/meryl \
      -Dt \
      -n 9164 \
      -s ./BC69.ms22 \
    > ./BC69.ms22.frequentMers.fasta \
    2> ./BC69.ms22.frequentMers.fasta.err

-- Finished on Sun May 13 09:12:40 2018 (lickety-split) with 549693.778 GB free disk space
----------------------------------------
-- Reset utgOvlMerThreshold from auto to 9164.
--
-- Found 17723465 22-mers; 668969 distinct and 445313 unique.  Largest count 13905.
--
-- OVERLAPPER (normal) (assembly) erate=0.144
--
----------------------------------------
-- Starting command on Sun May 13 09:12:40 2018 with 549693.778 GB free disk space

    cd unitigging/1-overlapper
    /home/dct7/canu/Linux-amd64/bin/overlapInCorePartition \
     -S  ../../BC69.seqStore \
     -hl 85312000 \
     -rl 1000000000 \
     -ol 500 \
     -o  ./BC69.partition \
    > ./BC69.partition.err 2>&1

-- Finished on Sun May 13 09:12:40 2018 (fuirously fast) with 549693.778 GB free disk space
----------------------------------------
--
-- Configured 1 overlapInCore jobs.
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'utgovl' concurrent execution on Sun May 13 09:12:40 2018 with 549693.778 GB free disk space (1 processes; 4 concurrently)

    cd unitigging/1-overlapper
    ./overlap.sh 1 > ./overlap.000001.out 2>&1

-- Finished on Sun May 13 11:18:17 2018 (7537 seconds) with 548882.974 GB free disk space
----------------------------------------
-- Found 1 overlapInCore output files.
--
-- overlapInCore compute 'unitigging/1-overlapper':
--   kmer hits
--     with no overlap         36524112    36524112 +- 0
--     with an overlap         45052343    45052343 +- 0
--
--   overlaps                  45052343    45052343 +- 0
--     contained               14481802    14481802 +- 0
--     dovetail                30570541    30570541 +- 0
--
--   overlaps rejected
--     multiple per pair              0           0 +- 0
--     bad short window               0           0 +- 0
--     bad long window                0           0 +- 0
----------------------------------------
-- Starting command on Sun May 13 11:18:17 2018 with 548882.974 GB free disk space

    cd unitigging
    /home/dct7/canu/Linux-amd64/bin/ovStoreConfig \
     -S ../BC69.seqStore \
     -M 4 \
     -L ./1-overlapper/ovljob.files \
     -create ./BC69.ovlStore.config \
     > ./BC69.ovlStore.config.txt \
    2> ./BC69.ovlStore.config.err

-- Finished on Sun May 13 11:18:17 2018 (like a bat out of hell) with 548882.974 GB free disk space
----------------------------------------
--
-- Creating overlap store unitigging/BC69.ovlStore using:
--      1 bucket
--      1 slice
--        using at most 3 GB memory each
----------------------------------------
-- Starting command on Sun May 13 11:18:17 2018 with 548882.974 GB free disk space

    cd unitigging
    /home/dct7/canu/Linux-amd64/bin/ovStoreBuild \
     -O  ./BC69.ovlStore.BUILDING \
     -S ../BC69.seqStore \
     -C  ./BC69.ovlStore.config \
     > ./BC69.ovlStore.err 2>&1

-- Finished on Sun May 13 11:18:46 2018 (29 seconds) with 548879.405 GB free disk space
----------------------------------------
-- Checking store.
----------------------------------------
-- Starting command on Sun May 13 11:18:46 2018 with 548879.405 GB free disk space

    cd unitigging
    /home/dct7/canu/Linux-amd64/bin/ovStoreDump \
     -S ../BC69.seqStore \
     -O  ./BC69.ovlStore \
     -counts \
     > ./BC69.ovlStore/counts.dat 2> ./BC69.ovlStore/counts.err

-- Finished on Sun May 13 11:18:46 2018 (in the blink of an eye) with 548879.405 GB free disk space
----------------------------------------
--
-- Overlap store 'unitigging/BC69.ovlStore' successfully constructed.
-- Found 90104686 overlaps for 9493 reads; 25109 reads have no overlaps.
--
--
-- Purged 0.529 GB in 3 overlap output files.
----------------------------------------
-- Starting command on Sun May 13 11:18:47 2018 with 548879.405 GB free disk space

    cd unitigging
    /home/dct7/canu/Linux-amd64/bin/ovStoreStats \
     -C 8961 \
     -S ../BC69.seqStore \
     -O  ./BC69.ovlStore \
     -o  ./BC69.ovlStore \
     > ./BC69.ovlStore.summary.err 2>&1
sh: line 5: 93784 Segmentation fault      /home/dct7/canu/Linux-amd64/bin/ovStoreStats -C 8961 -S ../BC69.seqStore -O ./BC69.ovlStore -o ./BC69.ovlStore > ./BC69.ovlStore.summary.err 2>&1

-- Finished on Sun May 13 11:19:07 2018 (20 seconds) with 548878.342 GB free disk space
----------------------------------------

ERROR:
ERROR:  Failed with exit code 139.  (rc=35584)
ERROR:
--
-- WARNING: failed to generate statistics for the overlap store; no summary will appear in report.
--
----------------------------------------
--
-- Loading read lengths.
-- Loading number of overlaps per read.
--
-- Configure RED for 8gb memory.
--                   Batches of at most (unlimited) reads.
--                                      500000000 bases.
--                   Expecting evidence of at most 536870912 bases per iteration.
--
--           Total                                               Reads                 Olaps Evidence
--    Job   Memory      Read Range         Reads        Bases   Memory        Olaps   Memory   Memory  (Memory in MB)
--   ---- -------- ------------------- --------- ------------ -------- ------------ -------- --------
--      1  4308.58         1-34602          9493     17922818   205.41     90104686  1031.17  1024.00
--   ---- -------- ------------------- --------- ------------ -------- ------------ -------- --------
--                                                   17922818              90104686
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'red' concurrent execution on Sun May 13 11:19:08 2018 with 548878.342 GB free disk space (1 processes; 8 concurrently)

    cd unitigging/3-overlapErrorAdjustment
    ./red.sh 1 > ./red.000001.out 2>&1

-- Finished on Sun May 13 12:05:22 2018 (2774 seconds) with 548651.308 GB free disk space
----------------------------------------
-- Found 1 read error detection output files.
--
-- Loading read lengths.
-- Loading number of overlaps per read.
--
-- Configure OEA for 4gb memory.
--                   Batches of at most (unlimited) reads.
--                                      300000000 bases.
--
--           Total                                               Reads                 Olaps  Adjusts
--    Job   Memory      Read Range         Reads        Bases   Memory        Olaps   Memory   Memory  (Memory in MB)
--   ---- -------- ------------------- --------- ------------ -------- ------------ -------- --------
--      1  4096.29         1-17005          5899     11130586    18.79     55992481  1708.75   320.74
--      2  3424.35     17006-34602          3594      6792232    14.59     34112205  1041.02   320.74
--   ---- -------- ------------------- --------- ------------ -------- ------------ -------- --------
--                                                   17922818              90104686
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'oea' concurrent execution on Sun May 13 12:05:22 2018 with 548651.308 GB free disk space (2 processes; 32 concurrently)

    cd unitigging/3-overlapErrorAdjustment
    ./oea.sh 1 > ./oea.000001.out 2>&1
    ./oea.sh 2 > ./oea.000002.out 2>&1

-- Finished on Sun May 13 13:41:12 2018 (5750 seconds) with 548375.693 GB free disk space
----------------------------------------
-- Found 2 overlap error adjustment output files.
----------------------------------------
-- Starting command on Sun May 13 13:41:12 2018 with 548375.693 GB free disk space

    cd unitigging/3-overlapErrorAdjustment
    /home/dct7/canu/Linux-amd64/bin/loadErates \
      -S ../../BC69.seqStore \
      -O ../BC69.ovlStore \
      -L ./oea.files \
    > ./oea.apply.err 2>&1

-- Finished on Sun May 13 13:41:13 2018 (one second) with 548375.693 GB free disk space
----------------------------------------
-- No report available.
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'bat' concurrent execution on Sun May 13 13:41:13 2018 with 548375.693 GB free disk space (1 processes; 1 concurrently)

    cd unitigging/4-unitigger
    ./unitigger.sh 1 > ./unitigger.000001.out 2>&1

-- Finished on Sun May 13 13:42:01 2018 (48 seconds) with 548373.298 GB free disk space
----------------------------------------
-- Unitigger finished successfully.
----------------------------------------
-- Starting command on Sun May 13 13:42:01 2018 with 548373.298 GB free disk space

    cd unitigging
    /home/dct7/canu/Linux-amd64/bin/tgStoreDump \
      -S ../BC69.seqStore \
      -T ./BC69.ctgStore 1 \
      -sizes -s 2000 \
    > ./BC69.ctgStore/seqDB.v001.sizes.txt

-- Finished on Sun May 13 13:42:01 2018 (in the blink of an eye) with 548373.298 GB free disk space
----------------------------------------
-- Found, in version 1, after unitig construction:
--   contigs:      1 sequences, total length 4881 bp (including 0 repeats of total length 0 bp).
--   bubbles:      0 sequences, total length 0 bp.
--   unassembled:  3529 sequences, total length 6617328 bp.
--
-- Contig sizes based on genome size 2kbp:
--
--            NG (bp)  LG (contigs)    sum (bp)
--         ----------  ------------  ----------
--     10        4881             1        4881
--     20        4881             1        4881
--     30        4881             1        4881
--     40        4881             1        4881
--     50        4881             1        4881
--     60        4881             1        4881
--     70        4881             1        4881
--     80        4881             1        4881
--     90        4881             1        4881
--    100        4881             1        4881
--    110        4881             1        4881
--    120        4881             1        4881
--    130        4881             1        4881
--    140        4881             1        4881
--    150        4881             1        4881
--    160        4881             1        4881
--    170        4881             1        4881
--    180        4881             1        4881
--    190        4881             1        4881
--    200        4881             1        4881
--    210        4881             1        4881
--    220        4881             1        4881
--    230        4881             1        4881
--    240        4881             1        4881
--
----------------------------------------
-- Starting command on Sun May 13 13:42:01 2018 with 548373.298 GB free disk space

    cd unitigging
    /home/dct7/canu/Linux-amd64/bin/sqStoreCreatePartition \
      -S ../BC69.seqStore \
      -T  ./BC69.ctgStore 1 \
      -b 15000 \
      -p 8 \
    > ./BC69.ctgStore/partitionedReads.log 2>&1

-- Finished on Sun May 13 13:42:01 2018 (in the blink of an eye) with 548373.298 GB free disk space
----------------------------------------
storePath 'unitigging/BC69.ctgStore'
storeName 'partitionedReads.seqStore'
----------------------------------------
-- Starting command on Sun May 13 13:42:01 2018 with 548373.298 GB free disk space

    cd unitigging
    /home/dct7/canu/Linux-amd64/bin/sqStoreCreatePartition \
      -S ../BC69.seqStore \
      -T  ./BC69.utgStore 1 \
      -b 15000 \
      -p 8 \
    > ./BC69.utgStore/partitionedReads.log 2>&1

-- Finished on Sun May 13 13:42:01 2018 (like a bat out of hell) with 548373.298 GB free disk space
----------------------------------------
storePath 'unitigging/BC69.utgStore'
storeName 'partitionedReads.seqStore'
-- Using slow alignment for consensus (iteration '0').
-- Configured 1 contig and 1 unitig consensus jobs.
--
--                            (tag)Concurrency
--                     (tag)Threads          |
--            (tag)Memory         |          |
--        (tag)         |         |          |     total usage     algorithm
--        -------  ------  --------   --------  -----------------  -----------------------------
-- Local: cns        1 GB    4 CPUs x   8 jobs     8 GB   32 CPUs  (consensus)
--
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'cns' concurrent execution on Sun May 13 13:42:01 2018 with 548373.298 GB free disk space (2 processes; 8 concurrently)

    cd unitigging/5-consensus
    ./consensus.sh 1 > ./consensus.000001.out 2>&1
    ./consensus.sh 2 > ./consensus.000002.out 2>&1

-- Finished on Sun May 13 13:42:10 2018 (9 seconds) with 548372.862 GB free disk space
----------------------------------------
-- Using slow alignment for consensus (iteration '1').
-- Configured 1 contig and 1 unitig consensus jobs.
-- All 2 consensus jobs finished successfully.
-- Using slow alignment for consensus (iteration '0').
-- Configured 1 contig and 1 unitig consensus jobs.
----------------------------------------
-- Starting command on Sun May 13 13:42:10 2018 with 548372.862 GB free disk space

    cd unitigging
    /home/dct7/canu/Linux-amd64/bin/tgStoreLoad \
      -S ../BC69.seqStore \
      -T  ./BC69.ctgStore 2 \
      -L ./5-consensus/ctgcns.files \
    > ./5-consensus/ctgcns.files.ctgStoreLoad.err 2>&1

-- Finished on Sun May 13 13:42:10 2018 (fast as lightning) with 548372.862 GB free disk space
----------------------------------------
----------------------------------------
-- Starting command on Sun May 13 13:42:10 2018 with 548372.862 GB free disk space

    cd unitigging
    /home/dct7/canu/Linux-amd64/bin/tgStoreLoad \
      -S ../BC69.seqStore \
      -T  ./BC69.utgStore 2 \
      -L ./5-consensus/utgcns.files \
    > ./5-consensus/utgcns.files.utgStoreLoad.err 2>&1

-- Finished on Sun May 13 13:42:10 2018 (lickety-split) with 548372.862 GB free disk space
----------------------------------------
-- Purging consensus output after loading to ctgStore and/or utgStore.
-- Purged 2 .cns outputs.
----------------------------------------
-- Starting command on Sun May 13 13:42:10 2018 with 548372.862 GB free disk space

    cd unitigging
    /home/dct7/canu/Linux-amd64/bin/tgStoreDump \
      -S ../BC69.seqStore \
      -T ./BC69.ctgStore 2 \
      -sizes -s 2000 \
    > ./BC69.ctgStore/seqDB.v002.sizes.txt

-- Finished on Sun May 13 13:42:10 2018 (fuirously fast) with 548372.862 GB free disk space
----------------------------------------
-- Found, in version 2, after consensus generation:
--   contigs:      1 sequences, total length 5374 bp (including 0 repeats of total length 0 bp).
--   bubbles:      0 sequences, total length 0 bp.
--   unassembled:  3529 sequences, total length 6617328 bp.
--
-- Contig sizes based on genome size 2kbp:
--
--            NG (bp)  LG (contigs)    sum (bp)
--         ----------  ------------  ----------
--     10        5374             1        5374
--     20        5374             1        5374
--     30        5374             1        5374
--     40        5374             1        5374
--     50        5374             1        5374
--     60        5374             1        5374
--     70        5374             1        5374
--     80        5374             1        5374
--     90        5374             1        5374
--    100        5374             1        5374
--    110        5374             1        5374
--    120        5374             1        5374
--    130        5374             1        5374
--    140        5374             1        5374
--    150        5374             1        5374
--    160        5374             1        5374
--    170        5374             1        5374
--    180        5374             1        5374
--    190        5374             1        5374
--    200        5374             1        5374
--    210        5374             1        5374
--    220        5374             1        5374
--    230        5374             1        5374
--    240        5374             1        5374
--    250        5374             1        5374
--    260        5374             1        5374
--
----------------------------------------
-- Starting command on Sun May 13 13:42:10 2018 with 548372.862 GB free disk space

    cd unitigging
    /home/dct7/canu/Linux-amd64/bin/tgStoreCoverageStat \
      -S ../BC69.seqStore \
      -T  ./BC69.ctgStore 2 \
      -s 2000 \
      -o ./BC69.ctgStore.coverageStat \
    > ./BC69.ctgStore.coverageStat.err 2>&1

-- Finished on Sun May 13 13:42:10 2018 (lickety-split) with 548372.862 GB free disk space
----------------------------------------
----------------------------------------
-- Starting command on Sun May 13 13:42:10 2018 with 548372.862 GB free disk space

    cd unitigging/4-unitigger
    ./alignGFA.sh \
    > alignGFA.err 2>&1

-- Finished on Sun May 13 13:42:10 2018 (fast as lightning) with 548372.862 GB free disk space
----------------------------------------
----------------------------------------
-- Starting command on Sun May 13 13:42:10 2018 with 548372.862 GB free disk space

    cd .
    /home/dct7/canu/Linux-amd64/bin/tgStoreDump \
      -S ./BC69.seqStore \
      -T ./unitigging/BC69.ctgStore 2 \
      -o ./BC69.contigs \
      -layout \
    > ./BC69.contigs.layout.err 2>&1

-- Finished on Sun May 13 13:42:11 2018 (one second) with 548372.862 GB free disk space
----------------------------------------
----------------------------------------
-- Starting command on Sun May 13 13:42:11 2018 with 548372.862 GB free disk space

    cd .
    /home/dct7/canu/Linux-amd64/bin/tgStoreDump \
      -S ./BC69.seqStore \
      -T ./unitigging/BC69.utgStore 2 \
      -o ./BC69.unitigs \
      -layout \
    > ./BC69.unitigs.layout.err 2>&1

-- Finished on Sun May 13 13:42:11 2018 (fast as lightning) with 548372.862 GB free disk space
----------------------------------------
----------------------------------------
-- Starting command on Sun May 13 13:42:11 2018 with 548372.862 GB free disk space

    cd .
    /home/dct7/canu/Linux-amd64/bin/tgStoreDump \
      -S ./BC69.seqStore \
      -T ./unitigging/BC69.ctgStore 2 \
      -consensus -fasta \
      -unassembled \
    > ./BC69.unassembled.fasta
    2> ./BC69.unassembled.err

-- Finished on Sun May 13 13:42:11 2018 (like a bat out of hell) with 548372.782 GB free disk space
----------------------------------------
----------------------------------------
-- Starting command on Sun May 13 13:42:11 2018 with 548372.782 GB free disk space

    cd .
    /home/dct7/canu/Linux-amd64/bin/tgStoreDump \
      -S ./BC69.seqStore \
      -T ./unitigging/BC69.ctgStore 2 \
      -consensus -fasta \
      -contigs \
    > ./BC69.contigs.fasta
    2> ./BC69.contigs.err

-- Finished on Sun May 13 13:42:11 2018 (fast as lightning) with 548372.782 GB free disk space
----------------------------------------
----------------------------------------
-- Starting command on Sun May 13 13:42:11 2018 with 548372.782 GB free disk space

    cd .
    /home/dct7/canu/Linux-amd64/bin/tgStoreDump \
      -S ./BC69.seqStore \
      -T ./unitigging/BC69.utgStore 2 \
      -consensus -fasta \
      -contigs \
    > ./BC69.unitigs.fasta
    2> ./BC69.unitigs.err

-- Finished on Sun May 13 13:42:11 2018 (like a bat out of hell) with 548372.782 GB free disk space
----------------------------------------
--
-- Assembly 'BC69' finished in '/n/scratch2/dct7/Run22/BC69'.
--
-- Summary saved in 'BC69.report'.
--
-- Sequences saved:
--   Contigs       -> 'BC69.contigs.fasta'
--   Unassembled   -> 'BC69.unassembled.fasta'
--   Unitigs       -> 'BC69.unitigs.fasta'
--
-- Read layouts saved:
--   Contigs       -> 'BC69.contigs.layout'.
--   Unitigs       -> 'BC69.unitigs.layout'.
--
-- Graphs saved:
--   Contigs       -> 'BC69.contigs.gfa'.
--   Unitigs       -> 'BC69.unitigs.gfa'.
--
-- Bye.
damientully commented 6 years ago

Seems to be related to a latest tip release as running the same command with v1.7 works and outputs the reads.

skoren commented 6 years ago

I think this is fixed in the latest tip if you update your code, that commit is about 50 behind the latest tip. You can always get the corrected or trimmed reads using the sqStoreDumpFASTQ command.

damientully commented 6 years ago

Thanks Sergey. Using that command worked in anycase.