search

marbl / canu

A single molecule sequence assembler for genomes large and small.
http://canu.readthedocs.io/
654 stars 179 forks source link

meryl-process failed, tried 2 times, giving up #2030

Closed kattweigh closed 2 years ago

kattweigh commented 2 years ago

Hi there,

I have been trying to run Canu 2.1.1 on some nanopore long reads using the script:

load_canu
canu gridOptions="-p safs" minThreads=48 -d /safs-data01/uqkweigh/long-read-assemblies/canu_P123/ -p P123 -genomeSize=8m -nanopore /safs-data02/dennislab/Public/Hoosfield/metagenomes/promethion/PromethION_FlipFlop_Garvin_27Mar20/coassemblies/allnanopore_P123.fq

I am connecting to a cluster through a Linux terminal. Despite running multiple times with high resources, it ends up crashing and generating the following error message:

meryl-process failed, tried 2 times, giving up ABORT: Don't panic, but a mostly harmless error occurred and Canu stopped. ABORT: Try restarting. If that doesn't work, ask for help

I receive some outputs including:

I have printed the canu.out file below.

Any suggestions would be greatly appreciated!

Cheers, Kattie

canu.out


Found perl:
   /usr/bin/perl
   This is perl 5, version 16, subversion 3 (v5.16.3) built for x86_64-linux-thread-multi

Found java:
   /usr/bin/java
   openjdk version "1.8.0_292"

Found canu:
   /home/safs-data02/dennislab/sw/canu/2.1.1/bin/canu
   canu 2.1.1

-- canu 2.1.1
--
-- CITATIONS
--
-- For 'standard' assemblies of PacBio or Nanopore reads:
--   Koren S, Walenz BP, Berlin K, Miller JR, Phillippy AM.
--   Canu: scalable and accurate long-read assembly via adaptive k-mer weighting and repeat separation.
--   Genome Res. 2017 May;27(5):722-736.
--   http://doi.org/10.1101/gr.215087.116
-- 
-- Read and contig alignments during correction and consensus use:
--   Šošic M, Šikic M.
--   Edlib: a C/C ++ library for fast, exact sequence alignment using edit distance.
--   Bioinformatics. 2017 May 1;33(9):1394-1395.
--   http://doi.org/10.1093/bioinformatics/btw753
-- 
-- Overlaps are generated using:
--   Berlin K, et al.
--   Assembling large genomes with single-molecule sequencing and locality-sensitive hashing.
--   Nat Biotechnol. 2015 Jun;33(6):623-30.
--   http://doi.org/10.1038/nbt.3238
-- 
--   Myers EW, et al.
--   A Whole-Genome Assembly of Drosophila.
--   Science. 2000 Mar 24;287(5461):2196-204.
--   http://doi.org/10.1126/science.287.5461.2196
-- 
-- Corrected read consensus sequences are generated using an algorithm derived from FALCON-sense:
--   Chin CS, et al.
--   Phased diploid genome assembly with single-molecule real-time sequencing.
--   Nat Methods. 2016 Dec;13(12):1050-1054.
--   http://doi.org/10.1038/nmeth.4035
-- 
-- Contig consensus sequences are generated using an algorithm derived from pbdagcon:
--   Chin CS, et al.
--   Nonhybrid, finished microbial genome assemblies from long-read SMRT sequencing data.
--   Nat Methods. 2013 Jun;10(6):563-9
--   http://doi.org/10.1038/nmeth.2474
-- 
-- CONFIGURE CANU
--
-- Detected Java(TM) Runtime Environment '1.8.0_292' (from 'java') with -d64 support.
-- Detected gnuplot version '4.6 patchlevel 2   ' (from 'gnuplot') and image format 'png'.
-- Detected 24 CPUs and 252 gigabytes of memory.
-- Detected Slurm with 'sinfo' binary in /usr/bin/sinfo.
-- Detected Slurm with task IDs up to 200000 allowed.
-- 
-- Found   2 hosts with  48 cores and 1511 GB memory under Slurm control.
-- Found  32 hosts with  16 cores and  188 GB memory under Slurm control.
-- Found  10 hosts with  28 cores and  251 GB memory under Slurm control.
-- Found   1 host  with   8 cores and   42 GB memory under Slurm control.
-- Found   6 hosts with  28 cores and  503 GB memory under Slurm control.
-- Found   2 hosts with  16 cores and   62 GB memory under Slurm control.
-- Found   3 hosts with  24 cores and   23 GB memory under Slurm control.
-- Found   2 hosts with   8 cores and   38 GB memory under Slurm control.
-- Found   2 hosts with  36 cores and  564 GB memory under Slurm control.
-- Found   3 hosts with  28 cores and  564 GB memory under Slurm control.
-- Found   1 host  with   2 cores and   15 GB memory under Slurm control.
-- Found  11 hosts with  20 cores and  251 GB memory under Slurm control.
-- Found   3 hosts with  12 cores and   23 GB memory under Slurm control.
-- Found   1 host  with   1 core  and    7 GB memory under Slurm control.
-- Found   3 hosts with  28 cores and  375 GB memory under Slurm control.
-- Found   2 hosts with  24 cores and   22 GB memory under Slurm control.
-- Found   1 host  with   8 cores and   93 GB memory under Slurm control.
-- Found   5 hosts with  32 cores and  187 GB memory under Slurm control.
-- Found  11 hosts with   8 cores and   46 GB memory under Slurm control.
-- Found   1 host  with  12 cores and   22 GB memory under Slurm control.
-- Found   2 hosts with  36 cores and  753 GB memory under Slurm control.
-- Found   1 host  with   2 cores and   30 GB memory under Slurm control.
-- Found   5 hosts with  24 cores and  251 GB memory under Slurm control.
--
--                         (tag)Threads
--                (tag)Memory         |
--        (tag)             |         |  algorithm
--        -------  ----------  --------  -----------------------------
-- Grid:  meryl     12.000 GB   48 CPUs  (k-mer counting)
-- Grid:  hap        8.000 GB   48 CPUs  (read-to-haplotype assignment)
-- Grid:  cormhap    6.000 GB   48 CPUs  (overlap detection with mhap)
-- Grid:  obtovl     4.000 GB   48 CPUs  (overlap detection)
-- Grid:  utgovl     4.000 GB   48 CPUs  (overlap detection)
-- Grid:  cor        8.000 GB   48 CPUs  (read correction)
-- Grid:  ovb        4.000 GB   48 CPUs  (overlap store bucketizer)
-- Grid:  ovs        8.000 GB   48 CPUs  (overlap store sorting)
-- Grid:  red       16.000 GB   48 CPUs  (read error detection)
-- Grid:  oea        8.000 GB   48 CPUs  (overlap error adjustment)
-- Grid:  bat       16.000 GB   48 CPUs  (contig construction with bogart)
-- Grid:  cns        -.--- GB   48 CPUs  (consensus)
--
-- In 'P123.seqStore', found Nanopore reads:
--   Nanopore:                 1
--
--   Raw:                      1
--
-- Generating assembly 'P123' in '/home/safs-data01/uqkweigh/long-read-assemblies/canu_P123':
--    - correct raw reads.
--    - trim corrected reads.
--    - assemble corrected and trimmed reads.
--
-- Parameters:
--
--  genomeSize        8000000
--
--  Overlap Generation Limits:
--    corOvlErrorRate 0.3200 ( 32.00%)
--    obtOvlErrorRate 0.1200 ( 12.00%)
--    utgOvlErrorRate 0.1200 ( 12.00%)
--
--  Overlap Processing Limits:
--    corErrorRate    0.5000 ( 50.00%)
--    obtErrorRate    0.1200 ( 12.00%)
--    utgErrorRate    0.1200 ( 12.00%)
--    cnsErrorRate    0.2000 ( 20.00%)
--
--
-- BEGIN CORRECTION
--
--
-- meryl-process failed, tried 2 times, giving up.
--

ABORT:
ABORT: canu 2.1.1
ABORT: Don't panic, but a mostly harmless error occurred and Canu stopped.
ABORT: Try restarting.  If that doesn't work, ask for help.
ABORT:
skoren commented 2 years ago

Most likely it's an issue with the grid config (see for example #1920). Post the contents of correction/0-mercounts/meryl-process*out files.

kattweigh commented 2 years ago

meryl-process.10982538_1.out

Found perl:
   /usr/bin/perl
   This is perl 5, version 16, subversion 3 (v5.16.3) built for x86_64-linux-thread-multi

Found java:
   /usr/bin/java
   openjdk version "1.8.0_292"

Found canu:
   /home/safs-data02/dennislab/sw/canu/2.1.1/bin/canu
   canu 2.1.1

Running job 1 based on SLURM_ARRAY_TASK_ID=1 and offset=0.

Found 1 command tree.

PROCESSING TREE #1 using 48 threads.
  opGreaterThan
    threshold=1
    opUnionSum
      ./P123.01.meryl
    output to P123.ms16.WORKING
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc

Failed with 'Aborted'; backtrace (libbacktrace):
utility/src/utility/system-stackTrace.C::83 in _Z17AS_UTL_catchCrashiP7siginfoPv()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
../../gcc-8.3.0/libstdc++-v3/libsupc++/vterminate.cc::95 in _ZN9__gnu_cxx27__verbose_terminate_handlerEv()
../../gcc-8.3.0/libstdc++-v3/libsupc++/eh_terminate.cc::47 in _ZN10__cxxabiv111__terminateEPFvvE()
../../gcc-8.3.0/libstdc++-v3/libsupc++/eh_call.cc::54 in __cxa_call_terminate()
../../gcc-8.3.0/libstdc++-v3/libsupc++/eh_personality.cc::676 in __gxx_personality_v0()
(null)::0 in (null)()
(null)::0 in (null)()
../../gcc-8.3.0/libstdc++-v3/libsupc++/eh_throw.cc::90 in __cxa_throw()
../../gcc-8.3.0/libstdc++-v3/libsupc++/new_op.cc::54 in _Znwm()
utility/src/utility/kmers-writer-stream.C::129 in _ZN17merylStreamWriter6addMerE8kmerTinyj()
meryl/src/meryl/merylOp-nextMer.C::658 in _ZN14merylOperation7nextMerEv()
meryl/src/meryl/meryl.C::250 in main._omp_fn.0()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
/var/spool/slurmd/job10982538/slurm_script: line 121: 28448 Aborted                 /home/safs-data02/dennislab/sw/canu/2.1.1/bin/meryl threads=48 memory=5 greater-than 1 output P123.ms16.WORKING union-sum ./P123.01.meryl
meryl merge failed.

meryl-process.10982542_1.out

Found perl:
   /usr/bin/perl
   This is perl 5, version 16, subversion 3 (v5.16.3) built for x86_64-linux-thread-multi

Found java:
   /usr/bin/java
   openjdk version "1.8.0_292"

Found canu:
   /home/safs-data02/dennislab/sw/canu/2.1.1/bin/canu
   canu 2.1.1

Running job 1 based on SLURM_ARRAY_TASK_ID=1 and offset=0.

Found 1 command tree.

PROCESSING TREE #1 using 48 threads.
  opGreaterThan
    threshold=1
    opUnionSum
      ./P123.01.meryl
    output to P123.ms16.WORKING
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
terminate called recursively

Failed with '
Failed with 'AbortedAborted'; backtrace (libbacktrace):
'; backtrace (libbacktrace):
terminate called recursively

Failed with 'Aborted'; backtrace (libbacktrace):

Failed with 'Segmentation fault'; backtrace (libbacktrace):

Failed with 'Segmentation fault'; backtrace (libbacktrace):
/var/spool/slurmd/job10982542/slurm_script: line 121: 28656 Segmentation fault      /home/safs-data02/dennislab/sw/canu/2.1.1/bin/meryl threads=48 memory=5 greater-than 1 output P123.ms16.WORKING union-sum ./P123.01.meryl
meryl merge failed.
brianwalenz commented 2 years ago

Reduce minThreads to 8 or maybe 16. I'm not sure exactly why it exhausts memory, but reducing the number of threads will reduce the number of buffers (working space) correspondingly.

kattweigh commented 2 years ago

Okay so I reduced threads to 16 and it got a little further to the overlapper step. See print out of canu.out below. It is now crashing due to: Mhap precompute jobs failed, retry

And in one of the precompute.11032365_205.out files it says:

There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 312 bytes for [thread 47135982073600 also had an error]AllocateHeap

# An error report file with more information is saved as:
# /home/safs-data01/uqkweigh/long-read-assemblies/canu_123/correction/1-overlapper/blocks/hs_err_pid30362.log
Mhap failed.

I saw on another issue post that this can be fixed changing JAVA OPTS? Is that correct? If so, how do I adjust this parameter?

**canu.out**

Found perl:
   /usr/bin/perl
   This is perl 5, version 16, subversion 3 (v5.16.3) built for x86_64-linux-thread-multi

Found java:
   /usr/bin/java
   openjdk version "1.8.0_292"

Found canu:
   /home/safs-data02/dennislab/sw/canu/2.1.1/bin/canu
   canu 2.1.1

-- canu 2.1.1
--
-- CITATIONS
--
-- For 'standard' assemblies of PacBio or Nanopore reads:
--   Koren S, Walenz BP, Berlin K, Miller JR, Phillippy AM.
--   Canu: scalable and accurate long-read assembly via adaptive k-mer weighting and repeat separation.
--   Genome Res. 2017 May;27(5):722-736.
--   http://doi.org/10.1101/gr.215087.116
-- 
-- Read and contig alignments during correction and consensus use:
--   Šošic M, Šikic M.
--   Edlib: a C/C ++ library for fast, exact sequence alignment using edit distance.
--   Bioinformatics. 2017 May 1;33(9):1394-1395.
--   http://doi.org/10.1093/bioinformatics/btw753
-- 
-- Overlaps are generated using:
--   Berlin K, et al.
--   Assembling large genomes with single-molecule sequencing and locality-sensitive hashing.
--   Nat Biotechnol. 2015 Jun;33(6):623-30.
--   http://doi.org/10.1038/nbt.3238
-- 
--   Myers EW, et al.
--   A Whole-Genome Assembly of Drosophila.
--   Science. 2000 Mar 24;287(5461):2196-204.
--   http://doi.org/10.1126/science.287.5461.2196
-- 
-- Corrected read consensus sequences are generated using an algorithm derived from FALCON-sense:
--   Chin CS, et al.
--   Phased diploid genome assembly with single-molecule real-time sequencing.
--   Nat Methods. 2016 Dec;13(12):1050-1054.
--   http://doi.org/10.1038/nmeth.4035
-- 
-- Contig consensus sequences are generated using an algorithm derived from pbdagcon:
--   Chin CS, et al.
--   Nonhybrid, finished microbial genome assemblies from long-read SMRT sequencing data.
--   Nat Methods. 2013 Jun;10(6):563-9
--   http://doi.org/10.1038/nmeth.2474
-- 
-- CONFIGURE CANU
--
-- Detected Java(TM) Runtime Environment '1.8.0_292' (from 'java') with -d64 support.
-- Detected gnuplot version '4.6 patchlevel 2   ' (from 'gnuplot') and image format 'png'.
-- Detected 24 CPUs and 252 gigabytes of memory.
-- Detected Slurm with 'sinfo' binary in /usr/bin/sinfo.
-- Detected Slurm with task IDs up to 200000 allowed.
-- 
-- Found   2 hosts with  48 cores and 1511 GB memory under Slurm control.
-- Found  32 hosts with  16 cores and  188 GB memory under Slurm control.
-- Found  10 hosts with  28 cores and  251 GB memory under Slurm control.
-- Found   1 host  with   8 cores and   42 GB memory under Slurm control.
-- Found   6 hosts with  28 cores and  503 GB memory under Slurm control.
-- Found   2 hosts with  16 cores and   62 GB memory under Slurm control.
-- Found   3 hosts with  24 cores and   23 GB memory under Slurm control.
-- Found   2 hosts with   8 cores and   38 GB memory under Slurm control.
-- Found   2 hosts with  36 cores and  564 GB memory under Slurm control.
-- Found   3 hosts with  28 cores and  564 GB memory under Slurm control.
-- Found   1 host  with   2 cores and   15 GB memory under Slurm control.
-- Found  11 hosts with  20 cores and  251 GB memory under Slurm control.
-- Found   3 hosts with  12 cores and   23 GB memory under Slurm control.
-- Found   1 host  with   1 core  and    7 GB memory under Slurm control.
-- Found   3 hosts with  28 cores and  375 GB memory under Slurm control.
-- Found   2 hosts with  24 cores and   22 GB memory under Slurm control.
-- Found   1 host  with   8 cores and   93 GB memory under Slurm control.
-- Found   5 hosts with  32 cores and  187 GB memory under Slurm control.
-- Found  11 hosts with   8 cores and   46 GB memory under Slurm control.
-- Found   1 host  with  12 cores and   22 GB memory under Slurm control.
-- Found   2 hosts with  36 cores and  753 GB memory under Slurm control.
-- Found   1 host  with   2 cores and   30 GB memory under Slurm control.
-- Found   5 hosts with  24 cores and  251 GB memory under Slurm control.
--
--                         (tag)Threads
--                (tag)Memory         |
--        (tag)             |         |  algorithm
--        -------  ----------  --------  -----------------------------
-- Grid:  meryl     12.000 GB   16 CPUs  (k-mer counting)
-- Grid:  hap        8.000 GB   16 CPUs  (read-to-haplotype assignment)
-- Grid:  cormhap    6.000 GB   16 CPUs  (overlap detection with mhap)
-- Grid:  obtovl     4.000 GB   16 CPUs  (overlap detection)
-- Grid:  utgovl     4.000 GB   16 CPUs  (overlap detection)
-- Grid:  cor        8.000 GB   16 CPUs  (read correction)
-- Grid:  ovb        4.000 GB   16 CPUs  (overlap store bucketizer)
-- Grid:  ovs        8.000 GB   16 CPUs  (overlap store sorting)
-- Grid:  red       16.000 GB   16 CPUs  (read error detection)
-- Grid:  oea        8.000 GB   16 CPUs  (overlap error adjustment)
-- Grid:  bat       16.000 GB   16 CPUs  (contig construction with bogart)
-- Grid:  cns        -.--- GB   16 CPUs  (consensus)
--
-- In 'P123.seqStore', found Nanopore reads:
--   Nanopore:                 1
--
--   Raw:                      1
--
-- Generating assembly 'P123' in '/home/safs-data01/uqkweigh/long-read-assemblies/canu_123':
--    - correct raw reads.
--    - trim corrected reads.
--    - assemble corrected and trimmed reads.
--
-- Parameters:
--
--  genomeSize        8000000
--
--  Overlap Generation Limits:
--    corOvlErrorRate 0.3200 ( 32.00%)
--    obtOvlErrorRate 0.1200 ( 12.00%)
--    utgOvlErrorRate 0.1200 ( 12.00%)
--
--  Overlap Processing Limits:
--    corErrorRate    0.5000 ( 50.00%)
--    obtErrorRate    0.1200 ( 12.00%)
--    utgErrorRate    0.1200 ( 12.00%)
--    cnsErrorRate    0.2000 ( 20.00%)
--
--
-- BEGIN CORRECTION
--
--
-- OVERLAPPER (mhap) (correction) complete, not rewriting scripts.
--
--
-- Mhap precompute jobs failed, tried 2 times, giving up.
--   job correction/1-overlapper/blocks/000001.dat FAILED.
--   job correction/1-overlapper/blocks/000002.dat FAILED.
--   job correction/1-overlapper/blocks/000003.dat FAILED.
--   job correction/1-overlapper/blocks/000004.dat FAILED.
--   job correction/1-overlapper/blocks/000005.dat FAILED.
--   job correction/1-overlapper/blocks/000006.dat FAILED.
--   job correction/1-overlapper/blocks/000007.dat FAILED.
--   job correction/1-overlapper/blocks/000008.dat FAILED.
--   job correction/1-overlapper/blocks/000009.dat FAILED.
--   job correction/1-overlapper/blocks/000010.dat FAILED.
--   job correction/1-overlapper/blocks/000011.dat FAILED.
--   job correction/1-overlapper/blocks/000012.dat FAILED.
--   job correction/1-overlapper/blocks/000013.dat FAILED.
--   job correction/1-overlapper/blocks/000014.dat FAILED.
--   job correction/1-overlapper/blocks/000015.dat FAILED.
--   job correction/1-overlapper/blocks/000016.dat FAILED.
--   job correction/1-overlapper/blocks/000017.dat FAILED.
--   job correction/1-overlapper/blocks/000018.dat FAILED.
--   job correction/1-overlapper/blocks/000019.dat FAILED.
--   job correction/1-overlapper/blocks/000020.dat FAILED.
--   job correction/1-overlapper/blocks/000021.dat FAILED.
--   job correction/1-overlapper/blocks/000022.dat FAILED.
--   job correction/1-overlapper/blocks/000023.dat FAILED.
--   job correction/1-overlapper/blocks/000024.dat FAILED.
--   job correction/1-overlapper/blocks/000025.dat FAILED.
--   job correction/1-overlapper/blocks/000026.dat FAILED.
--   job correction/1-overlapper/blocks/000027.dat FAILED.
--   job correction/1-overlapper/blocks/000028.dat FAILED.
--   job correction/1-overlapper/blocks/000029.dat FAILED.
--   job correction/1-overlapper/blocks/000030.dat FAILED.
--   job correction/1-overlapper/blocks/000031.dat FAILED.
--   job correction/1-overlapper/blocks/000032.dat FAILED.
--   job correction/1-overlapper/blocks/000033.dat FAILED.
--   job correction/1-overlapper/blocks/000034.dat FAILED.
--   job correction/1-overlapper/blocks/000035.dat FAILED.
--   job correction/1-overlapper/blocks/000036.dat FAILED.
--   job correction/1-overlapper/blocks/000037.dat FAILED.
--   job correction/1-overlapper/blocks/000038.dat FAILED.
--   job correction/1-overlapper/blocks/000039.dat FAILED.
--   job correction/1-overlapper/blocks/000040.dat FAILED.
--   job correction/1-overlapper/blocks/000041.dat FAILED.
--   job correction/1-overlapper/blocks/000042.dat FAILED.
--   job correction/1-overlapper/blocks/000043.dat FAILED.
--   job correction/1-overlapper/blocks/000044.dat FAILED.
--   job correction/1-overlapper/blocks/000045.dat FAILED.
--   job correction/1-overlapper/blocks/000046.dat FAILED.
--   job correction/1-overlapper/blocks/000047.dat FAILED.
--   job correction/1-overlapper/blocks/000048.dat FAILED.
--   job correction/1-overlapper/blocks/000049.dat FAILED.
--   job correction/1-overlapper/blocks/000050.dat FAILED.
--   job correction/1-overlapper/blocks/000051.dat FAILED.
--   job correction/1-overlapper/blocks/000052.dat FAILED.
--   job correction/1-overlapper/blocks/000053.dat FAILED.
--   job correction/1-overlapper/blocks/000054.dat FAILED.
--   job correction/1-overlapper/blocks/000055.dat FAILED.
--   job correction/1-overlapper/blocks/000056.dat FAILED.
--   job correction/1-overlapper/blocks/000057.dat FAILED.
--   job correction/1-overlapper/blocks/000058.dat FAILED.
--   job correction/1-overlapper/blocks/000059.dat FAILED.
--   job correction/1-overlapper/blocks/000060.dat FAILED.
--   job correction/1-overlapper/blocks/000061.dat FAILED.
--   job correction/1-overlapper/blocks/000062.dat FAILED.
--   job correction/1-overlapper/blocks/000063.dat FAILED.
--   job correction/1-overlapper/blocks/000064.dat FAILED.
--   job correction/1-overlapper/blocks/000065.dat FAILED.
--   job correction/1-overlapper/blocks/000066.dat FAILED.
--   job correction/1-overlapper/blocks/000067.dat FAILED.
--   job correction/1-overlapper/blocks/000068.dat FAILED.
--   job correction/1-overlapper/blocks/000069.dat FAILED.
--   job correction/1-overlapper/blocks/000070.dat FAILED.
--   job correction/1-overlapper/blocks/000071.dat FAILED.
--   job correction/1-overlapper/blocks/000072.dat FAILED.
--   job correction/1-overlapper/blocks/000073.dat FAILED.
--   job correction/1-overlapper/blocks/000074.dat FAILED.
--   job correction/1-overlapper/blocks/000075.dat FAILED.
--   job correction/1-overlapper/blocks/000076.dat FAILED.
--   job correction/1-overlapper/blocks/000077.dat FAILED.
--   job correction/1-overlapper/blocks/000078.dat FAILED.
--   job correction/1-overlapper/blocks/000079.dat FAILED.
--   job correction/1-overlapper/blocks/000080.dat FAILED.
--   job correction/1-overlapper/blocks/000081.dat FAILED.
--   job correction/1-overlapper/blocks/000082.dat FAILED.
--   job correction/1-overlapper/blocks/000083.dat FAILED.
--   job correction/1-overlapper/blocks/000084.dat FAILED.
--   job correction/1-overlapper/blocks/000085.dat FAILED.
--   job correction/1-overlapper/blocks/000086.dat FAILED.
--   job correction/1-overlapper/blocks/000087.dat FAILED.
--   job correction/1-overlapper/blocks/000088.dat FAILED.
--   job correction/1-overlapper/blocks/000089.dat FAILED.
--   job correction/1-overlapper/blocks/000090.dat FAILED.
--   job correction/1-overlapper/blocks/000091.dat FAILED.
--   job correction/1-overlapper/blocks/000092.dat FAILED.
--   job correction/1-overlapper/blocks/000093.dat FAILED.
--   job correction/1-overlapper/blocks/000094.dat FAILED.
--   job correction/1-overlapper/blocks/000095.dat FAILED.
--   job correction/1-overlapper/blocks/000096.dat FAILED.
--   job correction/1-overlapper/blocks/000097.dat FAILED.
--   job correction/1-overlapper/blocks/000098.dat FAILED.
--   job correction/1-overlapper/blocks/000099.dat FAILED.
--   job correction/1-overlapper/blocks/000100.dat FAILED.
--   job correction/1-overlapper/blocks/000101.dat FAILED.
--   job correction/1-overlapper/blocks/000102.dat FAILED.
--   job correction/1-overlapper/blocks/000103.dat FAILED.
--   job correction/1-overlapper/blocks/000104.dat FAILED.
--   job correction/1-overlapper/blocks/000105.dat FAILED.
--   job correction/1-overlapper/blocks/000106.dat FAILED.
--   job correction/1-overlapper/blocks/000107.dat FAILED.
--   job correction/1-overlapper/blocks/000108.dat FAILED.
--   job correction/1-overlapper/blocks/000109.dat FAILED.
--   job correction/1-overlapper/blocks/000110.dat FAILED.
--   job correction/1-overlapper/blocks/000111.dat FAILED.
--   job correction/1-overlapper/blocks/000112.dat FAILED.
--   job correction/1-overlapper/blocks/000113.dat FAILED.
--   job correction/1-overlapper/blocks/000114.dat FAILED.
--   job correction/1-overlapper/blocks/000115.dat FAILED.
--   job correction/1-overlapper/blocks/000116.dat FAILED.
--   job correction/1-overlapper/blocks/000117.dat FAILED.
--   job correction/1-overlapper/blocks/000118.dat FAILED.
--   job correction/1-overlapper/blocks/000119.dat FAILED.
--   job correction/1-overlapper/blocks/000120.dat FAILED.
--   job correction/1-overlapper/blocks/000121.dat FAILED.
--   job correction/1-overlapper/blocks/000122.dat FAILED.
--   job correction/1-overlapper/blocks/000123.dat FAILED.
--   job correction/1-overlapper/blocks/000124.dat FAILED.
--   job correction/1-overlapper/blocks/000125.dat FAILED.
--   job correction/1-overlapper/blocks/000126.dat FAILED.
--   job correction/1-overlapper/blocks/000127.dat FAILED.
--   job correction/1-overlapper/blocks/000128.dat FAILED.
--   job correction/1-overlapper/blocks/000129.dat FAILED.
--   job correction/1-overlapper/blocks/000130.dat FAILED.
--   job correction/1-overlapper/blocks/000131.dat FAILED.
--   job correction/1-overlapper/blocks/000132.dat FAILED.
--   job correction/1-overlapper/blocks/000133.dat FAILED.
--   job correction/1-overlapper/blocks/000134.dat FAILED.
--   job correction/1-overlapper/blocks/000135.dat FAILED.
--   job correction/1-overlapper/blocks/000136.dat FAILED.
--   job correction/1-overlapper/blocks/000137.dat FAILED.
--   job correction/1-overlapper/blocks/000138.dat FAILED.
--   job correction/1-overlapper/blocks/000139.dat FAILED.
--   job correction/1-overlapper/blocks/000140.dat FAILED.
--   job correction/1-overlapper/blocks/000141.dat FAILED.
--   job correction/1-overlapper/blocks/000142.dat FAILED.
--   job correction/1-overlapper/blocks/000143.dat FAILED.
--   job correction/1-overlapper/blocks/000144.dat FAILED.
--   job correction/1-overlapper/blocks/000145.dat FAILED.
--   job correction/1-overlapper/blocks/000146.dat FAILED.
--   job correction/1-overlapper/blocks/000147.dat FAILED.
--   job correction/1-overlapper/blocks/000148.dat FAILED.
--   job correction/1-overlapper/blocks/000149.dat FAILED.
--   job correction/1-overlapper/blocks/000150.dat FAILED.
--   job correction/1-overlapper/blocks/000151.dat FAILED.
--   job correction/1-overlapper/blocks/000152.dat FAILED.
--   job correction/1-overlapper/blocks/000153.dat FAILED.
--   job correction/1-overlapper/blocks/000154.dat FAILED.
--   job correction/1-overlapper/blocks/000155.dat FAILED.
--   job correction/1-overlapper/blocks/000156.dat FAILED.
--   job correction/1-overlapper/blocks/000157.dat FAILED.
--   job correction/1-overlapper/blocks/000158.dat FAILED.
--   job correction/1-overlapper/blocks/000159.dat FAILED.
--   job correction/1-overlapper/blocks/000160.dat FAILED.
--   job correction/1-overlapper/blocks/000161.dat FAILED.
--   job correction/1-overlapper/blocks/000162.dat FAILED.
--   job correction/1-overlapper/blocks/000163.dat FAILED.
--   job correction/1-overlapper/blocks/000164.dat FAILED.
--   job correction/1-overlapper/blocks/000165.dat FAILED.
--   job correction/1-overlapper/blocks/000166.dat FAILED.
--   job correction/1-overlapper/blocks/000167.dat FAILED.
--   job correction/1-overlapper/blocks/000168.dat FAILED.
--   job correction/1-overlapper/blocks/000169.dat FAILED.
--   job correction/1-overlapper/blocks/000170.dat FAILED.
--   job correction/1-overlapper/blocks/000171.dat FAILED.
--   job correction/1-overlapper/blocks/000172.dat FAILED.
--   job correction/1-overlapper/blocks/000173.dat FAILED.
--   job correction/1-overlapper/blocks/000174.dat FAILED.
--   job correction/1-overlapper/blocks/000175.dat FAILED.
--   job correction/1-overlapper/blocks/000176.dat FAILED.
--   job correction/1-overlapper/blocks/000177.dat FAILED.
--   job correction/1-overlapper/blocks/000178.dat FAILED.
--   job correction/1-overlapper/blocks/000179.dat FAILED.
--   job correction/1-overlapper/blocks/000180.dat FAILED.
--   job correction/1-overlapper/blocks/000181.dat FAILED.
--   job correction/1-overlapper/blocks/000182.dat FAILED.
--   job correction/1-overlapper/blocks/000183.dat FAILED.
--   job correction/1-overlapper/blocks/000184.dat FAILED.
--   job correction/1-overlapper/blocks/000185.dat FAILED.
--   job correction/1-overlapper/blocks/000186.dat FAILED.
--   job correction/1-overlapper/blocks/000187.dat FAILED.
--   job correction/1-overlapper/blocks/000188.dat FAILED.
--   job correction/1-overlapper/blocks/000189.dat FAILED.
--   job correction/1-overlapper/blocks/000190.dat FAILED.
--   job correction/1-overlapper/blocks/000191.dat FAILED.
--   job correction/1-overlapper/blocks/000192.dat FAILED.
--   job correction/1-overlapper/blocks/000193.dat FAILED.
--   job correction/1-overlapper/blocks/000194.dat FAILED.
--   job correction/1-overlapper/blocks/000195.dat FAILED.
--   job correction/1-overlapper/blocks/000196.dat FAILED.
--   job correction/1-overlapper/blocks/000197.dat FAILED.
--   job correction/1-overlapper/blocks/000198.dat FAILED.
--   job correction/1-overlapper/blocks/000199.dat FAILED.
--   job correction/1-overlapper/blocks/000200.dat FAILED.
--   job correction/1-overlapper/blocks/000201.dat FAILED.
--   job correction/1-overlapper/blocks/000202.dat FAILED.
--   job correction/1-overlapper/blocks/000203.dat FAILED.
--   job correction/1-overlapper/blocks/000204.dat FAILED.
--   job correction/1-overlapper/blocks/000205.dat FAILED.
--

ABORT:
ABORT: canu 2.1.1
ABORT: Don't panic, but a mostly harmless error occurred and Canu stopped.
ABORT: Try restarting.  If that doesn't work, ask for help.
ABORT:
kattweigh commented 2 years ago

The /home/safs-data01/uqkweigh/long-read-assemblies/canu_123/correction/1-overlapper/blocks/hs_err_pid30362.log mentioned is printed below:

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 312 bytes for AllocateHeap
# Possible reasons:
#   The system is out of physical RAM or swap space
#   The process is running with CompressedOops enabled, and the Java Heap may be blocking the growth of the native heap
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
#   JVM is running with Zero Based Compressed Oops mode in which the Java heap is
#     placed in the first 32GB address space. The Java Heap base address is the
#     maximum limit for the native heap growth. Please use -XX:HeapBaseMinAddress
#     to set the Java Heap base and to place the Java Heap above 32GB virtual address.
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (allocation.inline.hpp:61), pid=30362, tid=0x00002adeb3577700
#
# JRE version: OpenJDK Runtime Environment (8.0_292-b10) (build 1.8.0_292-b10)
# Java VM: OpenJDK 64-Bit Server VM (25.292-b10 mixed mode linux-amd64 compressed oops)
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#

---------------  T H R E A D  ---------------

Current thread (0x00002ade4c4a6000):  JavaThread "pool-2-thread-8" [_thread_in_vm, id=30405, stack(0x00002adeb3477000,0x00002adeb3578000)]

Stack: [0x00002adeb3477000,0x00002adeb3578000],  sp=0x00002adeb35768a0,  free space=1022k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0xb65b9d]
V  [libjvm.so+0x5082ba]
V  [libjvm.so+0x2ff9d2]
V  [libjvm.so+0x2ffa65]
V  [libjvm.so+0x731d3a]
V  [libjvm.so+0x69bbc7]
V  [libjvm.so+0x69e877]
V  [libjvm.so+0x69c544]
V  [libjvm.so+0x69cb59]
V  [libjvm.so+0x733ec1]
V  [libjvm.so+0xb0d192]
V  [libjvm.so+0x93de52]
C  [libpthread.so.0+0x7ea5]

---------------  P R O C E S S  ---------------

Java Threads: ( => current thread )
  0x00002ade4c4b5000 JavaThread "pool-2-thread-16" [_thread_in_vm, id=30413, stack(0x00002adeb3c90000,0x00002adeb3d91000)]
  0x00002ade4c4b3000 JavaThread "pool-2-thread-15" [_thread_in_vm, id=30412, stack(0x00002adeb3b8d000,0x00002adeb3c8e000)]
  0x00002ade4c4b1000 JavaThread "pool-2-thread-14" [_thread_in_vm, id=30411, stack(0x00002adeb3a87000,0x00002adeb3b88000)]
  0x00002ade4c4af000 JavaThread "pool-2-thread-13" [_thread_in_vm, id=30410, stack(0x00002adeb3984000,0x00002adeb3a85000)]
  0x00002ade4c4ad800 JavaThread "pool-2-thread-12" [_thread_blocked, id=30409, stack(0x00002adeb3883000,0x00002adeb3984000)]
  0x00002ade4c4ab800 JavaThread "pool-2-thread-11" [_thread_in_vm, id=30408, stack(0x00002adeb377c000,0x00002adeb387d000)]
  0x00002ade4c4a9800 JavaThread "pool-2-thread-10" [_thread_in_vm, id=30407, stack(0x00002adeb367b000,0x00002adeb377c000)]
  0x00002ade4c4a7800 JavaThread "pool-2-thread-9" [_thread_in_vm, id=30406, stack(0x00002adeb357a000,0x00002adeb367b000)]
=>0x00002ade4c4a6000 JavaThread "pool-2-thread-8" [_thread_in_vm, id=30405, stack(0x00002adeb3477000,0x00002adeb3578000)]
  0x00002ade4c4a4000 JavaThread "pool-2-thread-7" [_thread_blocked, id=30404, stack(0x00002adeb3374000,0x00002adeb3475000)]
  0x00002ade4c4a2000 JavaThread "pool-2-thread-6" [_thread_blocked, id=30403, stack(0x00002adeb3272000,0x00002adeb3373000)]
  0x00002ade4c4a0800 JavaThread "pool-2-thread-5" [_thread_in_vm, id=30402, stack(0x00002adeb3170000,0x00002adeb3271000)]
  0x00002ade4c49e800 JavaThread "pool-2-thread-4" [_thread_in_vm, id=30401, stack(0x00002adeb306d000,0x00002adeb316e000)]
  0x00002ade4c49c800 JavaThread "pool-2-thread-3" [_thread_blocked, id=30400, stack(0x00002adeb2f6c000,0x00002adeb306d000)]
  0x00002ade4c49b000 JavaThread "pool-2-thread-2" [_thread_blocked, id=30399, stack(0x00002adeb2e6a000,0x00002adeb2f6b000)]
  0x00002ade4c499800 JavaThread "pool-2-thread-1" [_thread_blocked, id=30398, stack(0x00002adeb2d66000,0x00002adeb2e67000)]
  0x00002ade4c3ad800 JavaThread "Service Thread" daemon [_thread_blocked, id=30396, stack(0x00002adeb25ae000,0x00002adeb26af000)]
  0x00002ade4c39a800 JavaThread "C1 CompilerThread11" daemon [_thread_blocked, id=30395, stack(0x00002adeb2436000,0x00002adeb2536000)]
  0x00002ade4c398000 JavaThread "C1 CompilerThread10" daemon [_thread_blocked, id=30394, stack(0x00002adeb2327000,0x00002adeb2427000)]
  0x00002ade4c396000 JavaThread "C1 CompilerThread9" daemon [_thread_blocked, id=30393, stack(0x00002adeb2226000,0x00002adeb2326000)]
  0x00002ade4c393800 JavaThread "C1 CompilerThread8" daemon [_thread_blocked, id=30392, stack(0x00002adeb2125000,0x00002adeb2225000)]
  0x00002ade4c391800 JavaThread "C2 CompilerThread7" daemon [_thread_blocked, id=30391, stack(0x00002adeb2004000,0x00002adeb2104000)]
  0x00002ade4c38f000 JavaThread "C2 CompilerThread6" daemon [_thread_blocked, id=30390, stack(0x00002adeb1f03000,0x00002adeb2003000)]
  0x00002ade4c38c800 JavaThread "C2 CompilerThread5" daemon [_thread_blocked, id=30389, stack(0x00002adeb1e02000,0x00002adeb1f02000)]
  0x00002ade4c38a800 JavaThread "C2 CompilerThread4" daemon [_thread_blocked, id=30388, stack(0x00002adeb1d01000,0x00002adeb1e01000)]
  0x00002ade4c388000 JavaThread "C2 CompilerThread3" daemon [_thread_blocked, id=30387, stack(0x00002adeb1c00000,0x00002adeb1d00000)]
  0x00002ade4c386000 JavaThread "C2 CompilerThread2" daemon [_thread_blocked, id=30386, stack(0x00002adeb1aff000,0x00002adeb1bff000)]
  0x00002ade4c384000 JavaThread "C2 CompilerThread1" daemon [_thread_blocked, id=30385, stack(0x00002adeb19fe000,0x00002adeb1afe000)]
  0x00002ade4c381000 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=30384, stack(0x00002adeb18fd000,0x00002adeb19fd000)]
  0x00002ade4c373800 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=30383, stack(0x00002adeb17fb000,0x00002adeb18fc000)]
  0x00002ade4c346800 JavaThread "Finalizer" daemon [_thread_blocked, id=30382, stack(0x00002adeb16fa000,0x00002adeb17fb000)]
  0x00002ade4c342000 JavaThread "Reference Handler" daemon [_thread_blocked, id=30381, stack(0x00002adeb15f9000,0x00002adeb16fa000)]
  0x00002ade4c04b800 JavaThread "main" [_thread_blocked, id=30363, stack(0x00002ade49833000,0x00002ade49933000)]

Other Threads:
  0x00002ade4c338000 VMThread [stack: 0x00002adeb14f9000,0x00002adeb15f9000] [id=30380]
  0x00002ade4c3b2000 WatcherThread [stack: 0x00002adeb26b1000,0x00002adeb27b1000] [id=30397]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread:  ([mutex/lock_event])
[0x00002ade4c048730] SystemDictionary_lock - owner thread: 0x00002ade4c4a9800
[0x00002ade4c0489b0] JNIHandleBlockFreeList_lock - owner thread: 0x00002ade4c4a6000

heap address: 0x0000000666600000, size: 5530 MB, Compressed Oops mode: Zero based, Oop shift amount: 3
Narrow klass base: 0x0000000000000000, Narrow klass shift: 3
Compressed class space size: 1073741824 Address: 0x00000007c0000000

Heap:
 PSYoungGen      total 1651712K, used 376402K [0x000000074cd00000, 0x00000007c0000000, 0x00000007c0000000)
  eden space 1416192K, 26% used [0x000000074cd00000,0x0000000763c948b8,0x00000007a3400000)
  from space 235520K, 0% used [0x00000007b1a00000,0x00000007b1a00000,0x00000007c0000000)
  to   space 235520K, 0% used [0x00000007a3400000,0x00000007a3400000,0x00000007b1a00000)
 ParOldGen       total 3775488K, used 0K [0x0000000666600000, 0x000000074cd00000, 0x000000074cd00000)
  object space 3775488K, 0% used [0x0000000666600000,0x0000000666600000,0x000000074cd00000)
 Metaspace       used 4590K, capacity 4884K, committed 5120K, reserved 1056768K
  class space    used 454K, capacity 464K, committed 512K, reserved 1048576K

Card table byte_map: [0x00002ade5fe1f000,0x00002ade608ed000] byte_map_base: 0x00002ade5caec000

Marking Bits: (ParMarkBitMap*) 0x00002ade4b6ba5e0
 Begin Bits: [0x00002adea4000000, 0x00002adea9668000)
 End Bits:   [0x00002adea9668000, 0x00002adeaecd0000)

Polling page: 0x00002ade4981a000

CodeCache: size=245760Kb used=3006Kb max_used=3006Kb free=242753Kb
 bounds [0x00002ade50a5f000, 0x00002ade50d5f000, 0x00002ade5fa5f000]
 total_blobs=402 nmethods=123 adapters=189
 compilation: enabled

Compilation events (230 events):
Event: 0.097 Thread 0x00002ade4c39a800    1       3       java.lang.String::hashCode (55 bytes)
Event: 0.098 Thread 0x00002ade4c39a800 nmethod 1 0x00002ade50d11450 code [0x00002ade50d115c0, 0x00002ade50d11850]
Event: 0.099 Thread 0x00002ade4c396000    2       3       java.lang.Object::<init> (1 bytes)
Event: 0.099 Thread 0x00002ade4c393800    3       3       java.lang.String::charAt (29 bytes)
Event: 0.099 Thread 0x00002ade4c396000 nmethod 2 0x00002ade50d11110 code [0x00002ade50d11280, 0x00002ade50d113d0]
Event: 0.099 Thread 0x00002ade4c396000    4       3       java.lang.String::equals (81 bytes)
Event: 0.099 Thread 0x00002ade4c393800 nmethod 3 0x00002ade50d10b50 code [0x00002ade50d10cc0, 0x00002ade50d10f28]
Event: 0.100 Thread 0x00002ade4c398000    5       3       java.lang.String::length (6 bytes)
Event: 0.100 Thread 0x00002ade4c396000 nmethod 4 0x00002ade50d10290 code [0x00002ade50d10420, 0x00002ade50d10970]
Event: 0.100 Thread 0x00002ade4c398000 nmethod 5 0x00002ade50d0fe10 code [0x00002ade50d0ff80, 0x00002ade50d100f0]
Event: 0.106 Thread 0x00002ade4c39a800    6       3       java.util.jar.Attributes$Name::isValid (32 bytes)
Event: 0.106 Thread 0x00002ade4c393800    7       3       java.util.jar.Attributes$Name::isAlpha (30 bytes)
Event: 0.106 Thread 0x00002ade4c396000    8       1       java.lang.Object::<init> (1 bytes)
Event: 0.106 Thread 0x00002ade4c393800 nmethod 7 0x00002ade50d06350 code [0x00002ade50d064c0, 0x00002ade50d066f0]
Event: 0.106 Thread 0x00002ade4c396000 nmethod 8 0x00002ade50d12f10 code [0x00002ade50d13060, 0x00002ade50d13170]
Event: 0.107 Thread 0x00002ade4c39a800 nmethod 6 0x00002ade50d12a10 code [0x00002ade50d12ba0, 0x00002ade50d12e58]
Event: 0.108 Thread 0x00002ade4c398000   10       3       java.lang.Math::min (11 bytes)
Event: 0.108 Thread 0x00002ade4c393800   12       3       java.lang.String::indexOf (70 bytes)
Event: 0.108 Thread 0x00002ade4c396000   11       1       java.lang.ref.Reference::get (5 bytes)
Event: 0.108 Thread 0x00002ade4c398000 nmethod 10 0x00002ade50d122d0 code [0x00002ade50d12440, 0x00002ade50d125d0]
Event: 0.108 Thread 0x00002ade4c398000   13       3       java.lang.AbstractStringBuilder::ensureCapacityInternal (27 bytes)
Event: 0.109 Thread 0x00002ade4c396000 nmethod 11 0x00002ade50d12010 code [0x00002ade50d12160, 0x00002ade50d12270]
Event: 0.109 Thread 0x00002ade4c39a800   14       1       java.lang.ThreadLocal::access$400 (5 bytes)
Event: 0.109 Thread 0x00002ade4c393800 nmethod 12 0x00002ade50d14410 code [0x00002ade50d145a0, 0x00002ade50d14928]
Event: 0.109 Thread 0x00002ade4c39a800 nmethod 14 0x00002ade50d11cd0 code [0x00002ade50d11e20, 0x00002ade50d11f10]
Event: 0.109 Thread 0x00002ade4c398000 nmethod 13 0x00002ade50d13c50 code [0x00002ade50d13de0, 0x00002ade50d14258]
Event: 0.110 Thread 0x00002ade4c396000   15       3       sun.misc.ASCIICaseInsensitiveComparator::toLower (16 bytes)
Event: 0.110 Thread 0x00002ade4c393800   16       3       sun.misc.ASCIICaseInsensitiveComparator::isUpper (18 bytes)
Event: 0.111 Thread 0x00002ade4c393800 nmethod 16 0x00002ade50d138d0 code [0x00002ade50d13a40, 0x00002ade50d13bf0]
Event: 0.111 Thread 0x00002ade4c396000 nmethod 15 0x00002ade50d13450 code [0x00002ade50d135c0, 0x00002ade50d13810]
Event: 0.111 Thread 0x00002ade4c396000   17       3       java.lang.AbstractStringBuilder::append (50 bytes)
Event: 0.111 Thread 0x00002ade4c39a800   18       3       java.lang.String::getChars (62 bytes)
Event: 0.111 Thread 0x00002ade4c398000   19       3       java.util.Arrays::copyOfRange (63 bytes)
Event: 0.111 Thread 0x00002ade4c39a800 nmethod 18 0x00002ade50d14a50 code [0x00002ade50d14c00, 0x00002ade50d15038]
Event: 0.112 Thread 0x00002ade4c396000 nmethod 17 0x00002ade50d15250 code [0x00002ade50d15420, 0x00002ade50d15b78]
Event: 0.112 Thread 0x00002ade4c396000   20       3       java.lang.CharacterData::of (120 bytes)
Event: 0.112 Thread 0x00002ade4c393800   21       3       java.lang.CharacterDataLatin1::getProperties (11 bytes)
Event: 0.112 Thread 0x00002ade4c39a800   22       3       java.lang.Character::toUpperCaseEx (30 bytes)
Event: 0.112 Thread 0x00002ade4c396000 nmethod 20 0x00002ade50d15e90 code [0x00002ade50d16040, 0x00002ade50d165b0]
Event: 0.112 Thread 0x00002ade4c396000   23       3       java.lang.CharacterDataLatin1::toUpperCaseEx (71 bytes)
Event: 0.112 Thread 0x00002ade4c39a800 nmethod 22 0x00002ade50d16810 code [0x00002ade50d16980, 0x00002ade50d16bd8]
Event: 0.112 Thread 0x00002ade4c393800 nmethod 21 0x00002ade50d16c90 code [0x00002ade50d16e00, 0x00002ade50d16f90]
Event: 0.113 Thread 0x00002ade4c398000 nmethod 19 0x00002ade50d17090 code [0x00002ade50d172a0, 0x00002ade50d17dd8]
Event: 0.113 Thread 0x00002ade4c396000 nmethod 23 0x00002ade50d18310 code [0x00002ade50d18480, 0x00002ade50d18730]
Event: 0.116 Thread 0x00002ade4c39a800   24       3       java.lang.System::getSecurityManager (4 bytes)
Event: 0.117 Thread 0x00002ade4c39a800 nmethod 24 0x00002ade50d19a50 code [0x00002ade50d19ba0, 0x00002ade50d19cf0]
Event: 0.120 Thread 0x00002ade4c393800   25       3       java.lang.String::indexOf (7 bytes)
Event: 0.121 Thread 0x00002ade4c393800 nmethod 25 0x00002ade50d1aa10 code [0x00002ade50d1ab80, 0x00002ade50d1ad88]
Event: 0.122 Thread 0x00002ade4c398000   26       3       java.lang.String::startsWith (72 bytes)
Event: 0.123 Thread 0x00002ade4c398000 nmethod 26 0x00002ade50d1ae10 code [0x00002ade50d1af80, 0x00002ade50d1b2b0]
Event: 0.126 Thread 0x00002ade4c39a800   27       3       java.util.concurrent.ConcurrentHashMap::tabAt (21 bytes)
Event: 0.126 Thread 0x00002ade4c398000   29       3       java.util.HashMap::hash (20 bytes)
Event: 0.126 Thread 0x00002ade4c39a800 nmethod 27 0x00002ade50d1ba50 code [0x00002ade50d1bbc0, 0x00002ade50d1bdf0]
Event: 0.126 Thread 0x00002ade4c398000 nmethod 29 0x00002ade50d1be90 code [0x00002ade50d1c000, 0x00002ade50d1c248]
Event: 0.128 Thread 0x00002ade4c396000   30       3       java.lang.String::lastIndexOf (52 bytes)
Event: 0.128 Thread 0x00002ade4c396000 nmethod 30 0x00002ade50d1d290 code [0x00002ade50d1d420, 0x00002ade50d1d7a8]
Event: 0.130 Thread 0x00002ade4c393800   31       3       sun.misc.FDBigInteger::mult (64 bytes)
Event: 0.130 Thread 0x00002ade4c39a800   32       3       sun.misc.FDBigInteger::<init> (30 bytes)
Event: 0.130 Thread 0x00002ade4c398000   33       3       sun.misc.FDBigInteger::trimLeadingZeros (57 bytes)
Event: 0.130 Thread 0x00002ade4c396000   34       3       sun.misc.FDBigInteger::makeImmutable (6 bytes)
Event: 0.131 Thread 0x00002ade4c396000 nmethod 34 0x00002ade50d1df10 code [0x00002ade50d1e060, 0x00002ade50d1e1b0]
Event: 0.131 Thread 0x00002ade4c39a800 nmethod 32 0x00002ade50d1e210 code [0x00002ade50d1e380, 0x00002ade50d1e588]
Event: 0.131 Thread 0x00002ade4c393800 nmethod 31 0x00002ade50d1e690 code [0x00002ade50d1e800, 0x00002ade50d1ea90]
Event: 0.131 Thread 0x00002ade4c393800   35       3       sun.misc.FDBigInteger::mult (44 bytes)
Event: 0.131 Thread 0x00002ade4c398000 nmethod 33 0x00002ade50d1ed50 code [0x00002ade50d1eec0, 0x00002ade50d1f1f0]
Event: 0.131 Thread 0x00002ade4c39a800   36       3       java.lang.StringBuilder::append (8 bytes)
Event: 0.131 Thread 0x00002ade4c396000   37       3       java.lang.String::<init> (82 bytes)
Event: 0.131 Thread 0x00002ade4c39a800 nmethod 36 0x00002ade50d1f310 code [0x00002ade50d1f480, 0x00002ade50d1f608]
Event: 0.131 Thread 0x00002ade4c393800 nmethod 35 0x00002ade50d1f690 code [0x00002ade50d1f820, 0x00002ade50d1fc18]
Event: 0.131 Thread 0x00002ade4c396000 nmethod 37 0x00002ade50d1fd90 code [0x00002ade50d1ff60, 0x00002ade50d20458]
Event: 0.132 Thread 0x00002ade4c39a800   38       3       java.util.HashMap::getNode (148 bytes)
Event: 0.132 Thread 0x00002ade4c396000   39       1       java.util.ArrayList::size (5 bytes)
Event: 0.132 Thread 0x00002ade4c396000 nmethod 39 0x00002ade50d206d0 code [0x00002ade50d20820, 0x00002ade50d20930]
Event: 0.132 Thread 0x00002ade4c38c800   40       4       java.lang.String::charAt (29 bytes)
Event: 0.133 Thread 0x00002ade4c39a800 nmethod 38 0x00002ade50d20990 code [0x00002ade50d20b60, 0x00002ade50d21508]
Event: 0.133 Thread 0x00002ade4c396000   41       3       java.util.Arrays::copyOf (19 bytes)
Event: 0.133 Thread 0x00002ade4c398000   42       3       java.lang.String::startsWith (7 bytes)
Event: 0.134 Thread 0x00002ade4c398000 nmethod 42 0x00002ade50d217d0 code [0x00002ade50d21940, 0x00002ade50d21b48]
Event: 0.134 Thread 0x00002ade4c396000 nmethod 41 0x00002ade50d21bd0 code [0x00002ade50d21d60, 0x00002ade50d22088]
Event: 0.134 Thread 0x00002ade4c396000   43       1       java.lang.String::length (6 bytes)
Event: 0.134 Thread 0x00002ade4c38c800 nmethod 40 0x00002ade50d24e50 code [0x00002ade50d24fa0, 0x00002ade50d25078]
Event: 0.134 Thread 0x00002ade4c396000 nmethod 43 0x00002ade50d24b50 code [0x00002ade50d24ca0, 0x00002ade50d24db0]
Event: 0.135 Thread 0x00002ade4c393800   44       3       java.util.HashMap::get (23 bytes)
Event: 0.135 Thread 0x00002ade4c398000   45       1       java.net.URL::getHost (5 bytes)
Event: 0.135 Thread 0x00002ade4c398000 nmethod 45 0x00002ade50d24890 code [0x00002ade50d249e0, 0x00002ade50d24af0]
Event: 0.135 Thread 0x00002ade4c393800 nmethod 44 0x00002ade50d24190 code [0x00002ade50d24320, 0x00002ade50d24738]
Event: 0.138 Thread 0x00002ade4c396000   47       3       java.util.HashMap::putVal (300 bytes)
Event: 0.138 Thread 0x00002ade4c39a800   46       3       java.util.HashMap::put (13 bytes)
Event: 0.138 Thread 0x00002ade4c398000   48       3       java.util.HashMap$Node::<init> (26 bytes)
Event: 0.139 Thread 0x00002ade4c398000 nmethod 48 0x00002ade50d22ed0 code [0x00002ade50d23040, 0x00002ade50d23230]
Event: 0.139 Thread 0x00002ade4c398000   49       3       java.util.HashMap::newNode (13 bytes)
Event: 0.139 Thread 0x00002ade4c39a800 nmethod 46 0x00002ade50d22850 code [0x00002ade50d229e0, 0x00002ade50d22d98]
Event: 0.139 Thread 0x00002ade4c39a800   50       3       java.util.HashMap::afterNodeInsertion (1 bytes)
Event: 0.139 Thread 0x00002ade4c39a800 nmethod 50 0x00002ade50d22550 code [0x00002ade50d226a0, 0x00002ade50d227f0]
Event: 0.139 Thread 0x00002ade4c398000 nmethod 49 0x00002ade50d25150 code [0x00002ade50d252c0, 0x00002ade50d25570]
Event: 0.140 Thread 0x00002ade4c396000 nmethod 47 0x00002ade50d25690 code [0x00002ade50d25900, 0x00002ade50d26bb8]
Event: 0.141 Thread 0x00002ade4c393800   51       3       java.lang.String::substring (79 bytes)
Event: 0.142 Thread 0x00002ade4c393800 nmethod 51 0x00002ade50d27290 code [0x00002ade50d27460, 0x00002ade50d27918]
Event: 0.142 Thread 0x00002ade4c39a800   52       3       java.lang.AbstractStringBuilder::newCapacity (39 bytes)
Event: 0.142 Thread 0x00002ade4c39a800 nmethod 52 0x00002ade50d27b50 code [0x00002ade50d27ce0, 0x00002ade50d280c8]
Event: 0.142 Thread 0x00002ade4c39a800   53       3       java.util.BitSet::wordIndex (5 bytes)
Event: 0.143 Thread 0x00002ade4c398000   54       3       java.util.BitSet::checkInvariants (111 bytes)
Event: 0.143 Thread 0x00002ade4c39a800 nmethod 53 0x00002ade50d28210 code [0x00002ade50d28360, 0x00002ade50d28490]
Event: 0.143 Thread 0x00002ade4c39a800   55       3       java.lang.AbstractStringBuilder::<init> (12 bytes)
Event: 0.143 Thread 0x00002ade4c396000   56       3       java.util.BitSet::get (69 bytes)
Event: 0.143 Thread 0x00002ade4c393800   57       1       java.lang.StringCoding$StringDecoder::requestedCharsetName (5 bytes)
Event: 0.143 Thread 0x00002ade4c393800 nmethod 57 0x00002ade50d28890 code [0x00002ade50d289e0, 0x00002ade50d28af0]
Event: 0.143 Thread 0x00002ade4c39a800 nmethod 55 0x00002ade50d28b50 code [0x00002ade50d28cc0, 0x00002ade50d28f10]
Event: 0.143 Thread 0x00002ade4c398000 nmethod 54 0x00002ade50d28fd0 code [0x00002ade50d29140, 0x00002ade50d292d0]
Event: 0.144 Thread 0x00002ade4c393800   59       3       java.lang.Character::toLowerCase (6 bytes)
Event: 0.144 Thread 0x00002ade4c39a800   60       3       java.lang.Character::toLowerCase (9 bytes)
Event: 0.144 Thread 0x00002ade4c398000   61       3       java.lang.CharacterDataLatin1::toLowerCase (39 bytes)
Event: 0.144 Thread 0x00002ade4c39a800 nmethod 60 0x00002ade50d29350 code [0x00002ade50d294c0, 0x00002ade50d296f8]
Event: 0.144 Thread 0x00002ade4c396000 nmethod 56 0x00002ade50d297d0 code [0x00002ade50d299e0, 0x00002ade50d2a368]
Event: 0.144 Thread 0x00002ade4c396000   62       1       java.net.URL::getProtocol (5 bytes)
Event: 0.144 Thread 0x00002ade4c398000 nmethod 61 0x00002ade50d2a810 code [0x00002ade50d2a980, 0x00002ade50d2ac10]
Event: 0.144 Thread 0x00002ade4c398000   63       3       java.lang.StringBuilder::<init> (7 bytes)
Event: 0.144 Thread 0x00002ade4c393800 nmethod 59 0x00002ade50d2ad90 code [0x00002ade50d2af20, 0x00002ade50d2b1b8]
Event: 0.144 Thread 0x00002ade4c39a800   64       3       java.lang.StringBuilder::toString (17 bytes)
Event: 0.144 Thread 0x00002ade4c396000 nmethod 62 0x00002ade50d2b2d0 code [0x00002ade50d2b420, 0x00002ade50d2b530]
Event: 0.144 Thread 0x00002ade4c398000 nmethod 63 0x00002ade50d2b590 code [0x00002ade50d2b700, 0x00002ade50d2b9b0]
Event: 0.144 Thread 0x00002ade4c39a800 nmethod 64 0x00002ade50d2bad0 code [0x00002ade50d2bc40, 0x00002ade50d2be48]
Event: 0.145 Thread 0x00002ade4c393800   65       1       java.io.File::getPath (5 bytes)
Event: 0.145 Thread 0x00002ade4c398000   66       1       java.net.URL::getPath (5 bytes)
Event: 0.145 Thread 0x00002ade4c393800 nmethod 65 0x00002ade50d2bf10 code [0x00002ade50d2c060, 0x00002ade50d2c170]
Event: 0.145 Thread 0x00002ade4c393800   67       1       java.net.URL::getAuthority (5 bytes)
Event: 0.145 Thread 0x00002ade4c398000 nmethod 66 0x00002ade50d2c1d0 code [0x00002ade50d2c320, 0x00002ade50d2c430]
Event: 0.145 Thread 0x00002ade4c393800 nmethod 67 0x00002ade50d2c490 code [0x00002ade50d2c5e0, 0x00002ade50d2c6f0]
Event: 0.146 Thread 0x00002ade4c39a800   68       3       sun.util.locale.LocaleUtils::isUpper (18 bytes)
Event: 0.146 Thread 0x00002ade4c396000   69       3       java.lang.ref.SoftReference::get (29 bytes)
Event: 0.146 Thread 0x00002ade4c398000   70       3       java.util.concurrent.ConcurrentHashMap::setTabAt (19 bytes)
Event: 0.146 Thread 0x00002ade4c39a800 nmethod 68 0x00002ade50d2cad0 code [0x00002ade50d2cc40, 0x00002ade50d2ce10]
Event: 0.146 Thread 0x00002ade4c396000 nmethod 69 0x00002ade50d2ce90 code [0x00002ade50d2d000, 0x00002ade50d2d1f0]
Event: 0.146 Thread 0x00002ade4c398000 nmethod 70 0x00002ade50d2d250 code [0x00002ade50d2d3a0, 0x00002ade50d2d510]
Event: 0.147 Thread 0x00002ade4c393800   73       3       java.util.concurrent.ConcurrentHashMap::casTabAt (20 bytes)
Event: 0.147 Thread 0x00002ade4c39a800   74       3       java.util.concurrent.ConcurrentHashMap::spread (10 bytes)
Event: 0.148 Thread 0x00002ade4c396000   75       3       java.util.concurrent.ConcurrentHashMap$Node::<init> (26 bytes)
Event: 0.148 Thread 0x00002ade4c393800 nmethod 73 0x00002ade50d2d950 code [0x00002ade50d2daa0, 0x00002ade50d2dc30]
Event: 0.148 Thread 0x00002ade4c39a800 nmethod 74 0x00002ade50d2dcd0 code [0x00002ade50d2de20, 0x00002ade50d2df70]
Event: 0.148 Thread 0x00002ade4c396000 nmethod 75 0x00002ade50d2dfd0 code [0x00002ade50d2e140, 0x00002ade50d2e350]
Event: 0.148 Thread 0x00002ade4c398000   76       3       java.io.BufferedInputStream::getBufIfOpen (21 bytes)
Event: 0.148 Thread 0x00002ade4c393800   77  s    3       java.io.BufferedInputStream::read (49 bytes)
Event: 0.148 Thread 0x00002ade4c39a800   78       3       java.io.DataInputStream::readInt (72 bytes)
Event: 0.149 Thread 0x00002ade4c398000 nmethod 76 0x00002ade50d2e410 code [0x00002ade50d2e580, 0x00002ade50d2e7c8]
Event: 0.149 Thread 0x00002ade4c39a800 nmethod 78 0x00002ade50d2e890 code [0x00002ade50d2ea40, 0x00002ade50d2efa8]
Event: 0.149 Thread 0x00002ade4c393800 nmethod 77 0x00002ade50d2f1d0 code [0x00002ade50d2f380, 0x00002ade50d2f8f8]
Event: 0.150 Thread 0x00002ade4c396000   79       3       java.util.StringTokenizer::skipDelimiters (116 bytes)
Event: 0.150 Thread 0x00002ade4c398000   80       3       java.util.StringTokenizer::scanToken (179 bytes)
Event: 0.150 Thread 0x00002ade4c39a800   81       3       java.util.StringTokenizer::hasMoreTokens (29 bytes)
Event: 0.150 Thread 0x00002ade4c393800   82       3       java.util.StringTokenizer::nextToken (92 bytes)
Event: 0.151 Thread 0x00002ade4c39a800 nmethod 81 0x00002ade50d2fb10 code [0x00002ade50d2fc80, 0x00002ade50d2fe88]
Event: 0.151 Thread 0x00002ade4c39a800   83       3       java.util.HashSet::add (20 bytes)
Event: 0.151 Thread 0x00002ade4c393800 nmethod 82 0x00002ade50d2ff10 code [0x00002ade50d300c0, 0x00002ade50d304b8]
Event: 0.151 Thread 0x00002ade4c39a800 nmethod 83 0x00002ade50d30610 code [0x00002ade50d307a0, 0x00002ade50d30cb8]
Event: 0.151 Thread 0x00002ade4c396000 nmethod 79 0x00002ade50d30e90 code [0x00002ade50d310a0, 0x00002ade50d31c38]
Event: 0.153 Thread 0x00002ade4c398000 nmethod 80 0x00002ade50d320d0 code [0x00002ade50d32360, 0x00002ade50d33618]
Event: 0.155 Thread 0x00002ade4c391800   85       4       java.lang.String::hashCode (55 bytes)
Event: 0.155 Thread 0x00002ade4c393800   84   !   3       java.util.concurrent.ConcurrentHashMap::putVal (362 bytes)
Event: 0.155 Thread 0x00002ade4c39a800   86       3       java.lang.ref.Reference::<init> (25 bytes)
Event: 0.155 Thread 0x00002ade4c39a800 nmethod 86 0x00002ade50d33d90 code [0x00002ade50d33f00, 0x00002ade50d34150]
Event: 0.155 Thread 0x00002ade4c396000   87       3       java.util.concurrent.ConcurrentHashMap::addCount (292 bytes)
Event: 0.156 Thread 0x00002ade4c398000   89       3       java.lang.String::endsWith (17 bytes)
Event: 0.156 Thread 0x00002ade4c398000 nmethod 89 0x00002ade50d34a10 code [0x00002ade50d34b80, 0x00002ade50d34da8]
Event: 0.157 Thread 0x00002ade4c39a800   90       3       java.util.zip.ZStreamRef::address (5 bytes)
Event: 0.157 Thread 0x00002ade4c398000   91       3       java.lang.String::lastIndexOf (13 bytes)
Event: 0.157 Thread 0x00002ade4c39a800 nmethod 90 0x00002ade50d34ed0 code [0x00002ade50d35020, 0x00002ade50d35170]
Event: 0.158 Thread 0x00002ade4c39a800   93  s!   3       sun.misc.URLClassPath::getLoader (243 bytes)
Event: 0.158 Thread 0x00002ade4c398000 nmethod 91 0x00002ade50d351d0 code [0x00002ade50d35340, 0x00002ade50d35568]
Event: 0.158 Thread 0x00002ade4c398000   92  s    3       sun.misc.URLClassPath::getNextLoader (88 bytes)
Event: 0.158 Thread 0x00002ade4c396000 nmethod 87 0x00002ade50d35610 code [0x00002ade50d35820, 0x00002ade50d36328]
Event: 0.158 Thread 0x00002ade4c393800 nmethod 84 0x00002ade50d366d0 code [0x00002ade50d369a0, 0x00002ade50d38378]
Event: 0.159 Thread 0x00002ade4c398000 nmethod 92 0x00002ade50d39010 code [0x00002ade50d39200, 0x00002ade50d39a88]
Event: 0.160 Thread 0x00002ade4c391800 nmethod 85 0x00002ade50d3cc50 code [0x00002ade50d3cda0, 0x00002ade50d3cf18]
Event: 0.161 Thread 0x00002ade4c39a800 nmethod 93 0x00002ade50d3cfd0 code [0x00002ade50d3d400, 0x00002ade50d40048]
Event: 0.162 Thread 0x00002ade4c396000   95       1       java.net.URL::getRef (5 bytes)
Event: 0.162 Thread 0x00002ade4c396000 nmethod 95 0x00002ade50d3c610 code [0x00002ade50d3c760, 0x00002ade50d3c870]
Event: 0.163 Thread 0x00002ade4c393800   96       3       java.lang.ClassLoader::checkName (43 bytes)
Event: 0.163 Thread 0x00002ade4c398000   97       3       sun.misc.VM::allowArraySyntax (4 bytes)
Event: 0.163 Thread 0x00002ade4c39a800   98   !   3       sun.misc.URLClassPath$JarLoader::getResource (85 bytes)
Event: 0.163 Thread 0x00002ade4c398000 nmethod 97 0x00002ade50d3c310 code [0x00002ade50d3c460, 0x00002ade50d3c5b0]
Event: 0.163 Thread 0x00002ade4c393800 nmethod 96 0x00002ade50d3b550 code [0x00002ade50d3b700, 0x00002ade50d3be78]
Event: 0.163 Thread 0x00002ade4c393800  100       3       java.lang.String::<init> (10 bytes)
Event: 0.163 Thread 0x00002ade4c396000   99       1       java.net.URL::getFile (5 bytes)
Event: 0.164 Thread 0x00002ade4c396000 nmethod 99 0x00002ade50d3b290 code [0x00002ade50d3b3e0, 0x00002ade50d3b4f0]
Event: 0.164 Thread 0x00002ade4c393800 nmethod 100 0x00002ade50d3ae90 code [0x00002ade50d3b000, 0x00002ade50d3b1d0]
Event: 0.164 Thread 0x00002ade4c39a800 nmethod 98 0x00002ade50d41b50 code [0x00002ade50d41dc0, 0x00002ade50d42a28]
Event: 0.166 Thread 0x00002ade4c384000  101       4       java.lang.String::indexOf (70 bytes)
Event: 0.166 Thread 0x00002ade4c396000  102       3       java.lang.String::replace (127 bytes)
Event: 0.166 Thread 0x00002ade4c396000 nmethod 102 0x00002ade50d43190 code [0x00002ade50d43340, 0x00002ade50d439f0]
Event: 0.167 Thread 0x00002ade4c393800  103       1       java.util.zip.ZipFile::access$000 (5 bytes)
Event: 0.167 Thread 0x00002ade4c393800 nmethod 103 0x00002ade50d3a7d0 code [0x00002ade50d3a920, 0x00002ade50d3aa10]
Event: 0.168 Thread 0x00002ade4c39a800  104       3       java.lang.String::indexOf (166 bytes)
Event: 0.168 Thread 0x00002ade4c393800  105       3       java.nio.Buffer::checkIndex (22 bytes)
Event: 0.168 Thread 0x00002ade4c396000  106       3       java.nio.DirectLongBufferU::ix (10 bytes)
Event: 0.168 Thread 0x00002ade4c396000 nmethod 106 0x00002ade50d3a490 code [0x00002ade50d3a5e0, 0x00002ade50d3a750]
Event: 0.168 Thread 0x00002ade4c396000  107       1       java.security.ProtectionDomain::getCodeSource (5 bytes)
Event: 0.168 Thread 0x00002ade4c396000 nmethod 107 0x00002ade50d3a1d0 code [0x00002ade50d3a320, 0x00002ade50d3a430]
Event: 0.168 Thread 0x00002ade4c393800 nmethod 105 0x00002ade50d46b50 code [0x00002ade50d46cc0, 0x00002ade50d46f08]
Event: 0.168 Thread 0x00002ade4c384000 nmethod 101 0x00002ade50d46fd0 code [0x00002ade50d47120, 0x00002ade50d472b8]
Event: 0.169 Thread 0x00002ade4c39a800 nmethod 104 0x00002ade50d45f90 code [0x00002ade50d46140, 0x00002ade50d46790]
Event: 0.169 Thread 0x00002ade4c39a800  108       3       java.lang.Character::toUpperCase (6 bytes)
Event: 0.169 Thread 0x00002ade4c396000  109       3       java.lang.Character::toUpperCase (9 bytes)
Event: 0.169 Thread 0x00002ade4c398000  110       3       java.lang.CharacterDataLatin1::toUpperCase (53 bytes)
Event: 0.169 Thread 0x00002ade4c393800  111       3       java.util.concurrent.ConcurrentHashMap::putIfAbsent (8 bytes)
Event: 0.169 Thread 0x00002ade4c396000 nmethod 109 0x00002ade50d45b10 code [0x00002ade50d45c80, 0x00002ade50d45eb8]
Event: 0.169 Thread 0x00002ade4c39a800 nmethod 108 0x00002ade50d455d0 code [0x00002ade50d45760, 0x00002ade50d459f8]
Event: 0.169 Thread 0x00002ade4c393800 nmethod 111 0x00002ade50d451d0 code [0x00002ade50d45340, 0x00002ade50d45548]
Event: 0.169 Thread 0x00002ade4c398000 nmethod 110 0x00002ade50d44c10 code [0x00002ade50d44d80, 0x00002ade50d45050]
Event: 0.170 Thread 0x00002ade4c396000  112       3       sun.misc.MetaIndex::mayContain (51 bytes)
Event: 0.170 Thread 0x00002ade4c396000 nmethod 112 0x00002ade50d44150 code [0x00002ade50d44300, 0x00002ade50d44978]
Event: 0.173 Thread 0x00002ade4c393800  113       1       java.nio.Buffer::position (5 bytes)
Event: 0.173 Thread 0x00002ade4c393800 nmethod 113 0x00002ade50d47810 code [0x00002ade50d47960, 0x00002ade50d47a70]
Event: 0.174 Thread 0x00002ade4c39a800  114       1       sun.misc.URLClassPath$JarLoader::access$700 (5 bytes)
Event: 0.174 Thread 0x00002ade4c39a800 nmethod 114 0x00002ade50d47ad0 code [0x00002ade50d47c20, 0x00002ade50d47d10]
Event: 0.174 Thread 0x00002ade4c39a800  115       3       java.lang.String::equalsIgnoreCase (48 bytes)
Event: 0.174 Thread 0x00002ade4c398000  116       1       java.net.URL::getQuery (5 bytes)
Event: 0.174 Thread 0x00002ade4c398000 nmethod 116 0x00002ade50d47d90 code [0x00002ade50d47ee0, 0x00002ade50d47ff0]
Event: 0.175 Thread 0x00002ade4c39a800 nmethod 115 0x00002ade50d48050 code [0x00002ade50d481c0, 0x00002ade50d48528]
Event: 0.175 Thread 0x00002ade4c396000  117       3       sun.nio.cs.UTF_8$Encoder::encode (359 bytes)
Event: 0.176 Thread 0x00002ade4c396000 nmethod 117 0x00002ade50d48650 code [0x00002ade50d48860, 0x00002ade50d493c8]
Event: 0.177 Thread 0x00002ade4c393800  118       1       java.net.URL::getPort (5 bytes)
Event: 0.177 Thread 0x00002ade4c393800 nmethod 118 0x00002ade50d49cd0 code [0x00002ade50d49e20, 0x00002ade50d49f30]
Event: 0.184 Thread 0x00002ade4c398000  119   !   3       java.lang.ClassLoader::loadClass (122 bytes)
Event: 0.184 Thread 0x00002ade4c393800  120       3       java.lang.ClassLoader::getClassLoadingLock (35 bytes)
Event: 0.184 Thread 0x00002ade4c39a800  121       3       java.lang.ClassLoader::findLoadedClass (16 bytes)
Event: 0.185 Thread 0x00002ade4c39a800 nmethod 121 0x00002ade50d4a350 code [0x00002ade50d4a4e0, 0x00002ade50d4a718]
Event: 0.185 Thread 0x00002ade4c393800 nmethod 120 0x00002ade50d4a7d0 code [0x00002ade50d4a960, 0x00002ade50d4ad28]
Event: 0.189 Thread 0x00002ade4c398000 nmethod 119 0x00002ade50d4ae90 code [0x00002ade50d4b220, 0x00002ade50d4d578]
Event: 0.190 Thread 0x00002ade4c396000  123       1       java.lang.reflect.Field::getName (5 bytes)
Event: 0.191 Thread 0x00002ade4c396000 nmethod 123 0x00002ade50d4ed90 code [0x00002ade50d4eee0, 0x00002ade50d4eff0]

GC Heap History (0 events):
No events

Deoptimization events (0 events):
No events

Classes redefined (0 events):
No events

Internal exceptions (6 events):
Event: 0.053 Thread 0x00002ade4c04b800 Exception <a 'java/lang/NoSuchMethodError': Method sun.misc.Unsafe.defineClass(Ljava/lang/String;[BII)Ljava/lang/Class; name or signature does not match> (0x000000074cd07cc0) thrown at [/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64
Event: 0.053 Thread 0x00002ade4c04b800 Exception <a 'java/lang/NoSuchMethodError': Method sun.misc.Unsafe.prefetchRead(Ljava/lang/Object;J)V name or signature does not match> (0x000000074cd07fa8) thrown at [/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/openjdk/hotspot/s
Event: 0.144 Thread 0x00002ade4c04b800 Exception <a 'java/security/PrivilegedActionException'> (0x000000074ce0aa98) thrown at [/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/openjdk/hotspot/src/share/vm/prims/jvm.cpp, line 1530]
Event: 0.144 Thread 0x00002ade4c04b800 Exception <a 'java/security/PrivilegedActionException'> (0x000000074ce0ae90) thrown at [/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/openjdk/hotspot/src/share/vm/prims/jvm.cpp, line 1530]
Event: 0.145 Thread 0x00002ade4c04b800 Exception <a 'java/security/PrivilegedActionException'> (0x000000074ce0dd48) thrown at [/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/openjdk/hotspot/src/share/vm/prims/jvm.cpp, line 1530]
Event: 0.145 Thread 0x00002ade4c04b800 Exception <a 'java/security/PrivilegedActionException'> (0x000000074ce0e140) thrown at [/builddir/build/BUILD/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/openjdk/hotspot/src/share/vm/prims/jvm.cpp, line 1530]

Events (250 events):
Event: 0.150 loading class java/util/spi/CalendarDataProvider
Event: 0.150 loading class java/util/spi/CalendarDataProvider done
Event: 0.150 loading class sun/util/locale/provider/CurrencyNameProviderImpl
Event: 0.150 loading class sun/util/locale/provider/CurrencyNameProviderImpl done
Event: 0.150 loading class sun/util/locale/provider/SPILocaleProviderAdapter$1
Event: 0.150 loading class sun/util/locale/provider/SPILocaleProviderAdapter$1 done
Event: 0.150 loading class java/util/Currency$CurrencyNameGetter
Event: 0.150 loading class sun/util/locale/provider/LocaleServiceProviderPool$LocalizedObjectGetter
Event: 0.150 loading class sun/util/locale/provider/LocaleServiceProviderPool$LocalizedObjectGetter done
Event: 0.150 loading class java/util/Currency$CurrencyNameGetter done
Event: 0.150 loading class sun/util/resources/CurrencyNames
Event: 0.151 loading class sun/util/resources/LocaleNamesBundle
Event: 0.151 loading class sun/util/resources/OpenListResourceBundle
Event: 0.151 loading class sun/util/resources/OpenListResourceBundle done
Event: 0.151 loading class sun/util/resources/LocaleNamesBundle done
Event: 0.151 loading class sun/util/resources/CurrencyNames done
Event: 0.151 loading class sun/util/resources/en/CurrencyNames_en_US
Event: 0.151 loading class sun/util/resources/en/CurrencyNames_en_US done
Event: 0.151 loading class java/util/zip/DataFormatException
Event: 0.152 loading class java/util/zip/DataFormatException done
Event: 0.152 loading class java/io/EOFException
Event: 0.152 loading class java/io/EOFException done
Event: 0.154 loading class java/util/zip/CRC32
Event: 0.154 loading class java/util/zip/Checksum
Event: 0.154 loading class java/util/zip/Checksum done
Event: 0.154 loading class java/util/zip/CRC32 done
Event: 0.154 loading class org/apache/commons/compress/compressors/gzip/GzipParameters
Event: 0.154 loading class org/apache/commons/compress/compressors/gzip/GzipParameters done
Event: 0.155 loading class edu/umd/marbl/mhap/sketch/FrequencyCounts
Event: 0.155 loading class edu/umd/marbl/mhap/sketch/FrequencyCounts done
Event: 0.155 loading class java/util/concurrent/RejectedExecutionHandler
Event: 0.155 loading class java/util/concurrent/RejectedExecutionHandler done
Event: 0.156 loading class java/util/concurrent/BlockingQueue
Event: 0.156 loading class java/util/concurrent/BlockingQueue done
Event: 0.156 loading class it/unimi/dsi/fastutil/ints/IntOpenHashSet
Event: 0.156 loading class it/unimi/dsi/fastutil/ints/IntOpenHashSet done
Event: 0.156 loading class it/unimi/dsi/fastutil/Hash
Event: 0.156 loading class it/unimi/dsi/fastutil/Hash done
Event: 0.157 loading class it/unimi/dsi/fastutil/ints/AbstractIntSet
Event: 0.157 loading class it/unimi/dsi/fastutil/ints/AbstractIntSet done
Event: 0.157 loading class it/unimi/dsi/fastutil/ints/IntSet
Event: 0.157 loading class it/unimi/dsi/fastutil/ints/IntSet done
Event: 0.157 loading class it/unimi/dsi/fastutil/ints/IntCollection
Event: 0.157 loading class it/unimi/dsi/fastutil/ints/IntCollection done
Event: 0.157 loading class it/unimi/dsi/fastutil/ints/IntIterable
Event: 0.157 loading class it/unimi/dsi/fastutil/ints/IntIterable done
Event: 0.158 loading class it/unimi/dsi/fastutil/ints/AbstractIntCollection
Event: 0.158 loading class it/unimi/dsi/fastutil/ints/AbstractIntCollection done
Event: 0.162 loading class java/lang/UnsupportedOperationException
Event: 0.162 loading class java/lang/UnsupportedOperationException done
Event: 0.162 loading class java/lang/CloneNotSupportedException
Event: 0.163 loading class java/lang/CloneNotSupportedException done
Event: 0.163 loading class java/lang/InternalError
Event: 0.163 loading class java/lang/InternalError done
Event: 0.163 loading class it/unimi/dsi/fastutil/ints/IntIterator
Event: 0.163 loading class it/unimi/dsi/fastutil/ints/IntIterator done
Event: 0.163 loading class it/unimi/dsi/fastutil/HashCommon
Event: 0.163 loading class it/unimi/dsi/fastutil/HashCommon done
Event: 0.163 loading class java/lang/StrictMath
Event: 0.164 loading class java/lang/StrictMath done
Event: 0.164 loading class java/util/concurrent/atomic/AtomicReference
Event: 0.164 loading class java/util/concurrent/atomic/AtomicReference done
Event: 0.164 loading class java/util/regex/Pattern
Event: 0.164 loading class java/util/regex/Pattern done
Event: 0.164 loading class java/util/regex/Pattern$4
Event: 0.164 loading class java/util/regex/Pattern$Node
Event: 0.164 loading class java/util/regex/Pattern$Node done
Event: 0.164 loading class java/util/regex/Pattern$4 done
Event: 0.164 loading class java/util/regex/Pattern$LastNode
Event: 0.164 loading class java/util/regex/Pattern$LastNode done
Event: 0.164 loading class java/util/regex/Pattern$GroupHead
Event: 0.164 loading class java/util/regex/Pattern$GroupHead done
Event: 0.164 loading class java/util/regex/Pattern$Ctype
Event: 0.165 loading class java/util/regex/Pattern$BmpCharProperty
Event: 0.165 loading class java/util/regex/Pattern$CharProperty
Event: 0.165 loading class java/util/regex/Pattern$CharProperty done
Event: 0.165 loading class java/util/regex/Pattern$BmpCharProperty done
Event: 0.165 loading class java/util/regex/Pattern$Ctype done
Event: 0.165 loading class java/util/regex/Pattern$Curly
Event: 0.165 loading class java/util/regex/Pattern$Curly done
Event: 0.165 loading class java/util/regex/Pattern$Slice
Event: 0.165 loading class java/util/regex/Pattern$SliceNode
Event: 0.165 loading class java/util/regex/Pattern$SliceNode done
Event: 0.165 loading class java/util/regex/Pattern$Slice done
Event: 0.165 loading class java/util/regex/Pattern$Begin
Event: 0.165 loading class java/util/regex/Pattern$Begin done
Event: 0.165 loading class java/util/regex/Pattern$First
Event: 0.165 loading class java/util/regex/Pattern$First done
Event: 0.165 loading class java/util/regex/Pattern$Start
Event: 0.165 loading class java/util/regex/Pattern$Start done
Event: 0.165 loading class java/util/regex/Pattern$TreeInfo
Event: 0.165 loading class java/util/regex/Pattern$TreeInfo done
Event: 0.165 loading class java/util/regex/Matcher
Event: 0.165 loading class java/util/regex/MatchResult
Event: 0.165 loading class java/util/regex/MatchResult done
Event: 0.165 loading class java/util/regex/Matcher done
Event: 0.165 loading class java/util/regex/ASCII
Event: 0.165 loading class java/util/regex/ASCII done
Event: 0.165 loading class java/util/ArrayList$SubList
Event: 0.165 loading class java/util/ArrayList$SubList done
Event: 0.165 loading class java/util/ArrayList$SubList$1
Event: 0.165 loading class java/util/ListIterator
Event: 0.165 loading class java/util/ListIterator done
Event: 0.165 loading class java/util/ArrayList$SubList$1 done
Event: 0.166 loading class it/unimi/dsi/fastutil/longs/Long2DoubleOpenHashMap
Event: 0.166 loading class it/unimi/dsi/fastutil/longs/Long2DoubleOpenHashMap done
Event: 0.166 loading class it/unimi/dsi/fastutil/longs/AbstractLong2DoubleMap
Event: 0.166 loading class it/unimi/dsi/fastutil/longs/AbstractLong2DoubleMap done
Event: 0.166 loading class it/unimi/dsi/fastutil/longs/Long2DoubleMap
Event: 0.166 loading class it/unimi/dsi/fastutil/longs/Long2DoubleMap done
Event: 0.167 loading class it/unimi/dsi/fastutil/longs/Long2DoubleFunction
Event: 0.167 loading class it/unimi/dsi/fastutil/longs/Long2DoubleFunction done
Event: 0.167 loading class it/unimi/dsi/fastutil/Function
Event: 0.167 loading class it/unimi/dsi/fastutil/Function done
Event: 0.167 loading class it/unimi/dsi/fastutil/longs/AbstractLong2DoubleFunction
Event: 0.167 loading class it/unimi/dsi/fastutil/longs/AbstractLong2DoubleFunction done
Event: 0.168 loading class it/unimi/dsi/fastutil/doubles/DoubleCollection
Event: 0.168 loading class it/unimi/dsi/fastutil/doubles/DoubleCollection done
Event: 0.168 loading class it/unimi/dsi/fastutil/doubles/DoubleIterable
Event: 0.168 loading class it/unimi/dsi/fastutil/doubles/DoubleIterable done
Event: 0.168 loading class it/unimi/dsi/fastutil/longs/LongSet
Event: 0.168 loading class it/unimi/dsi/fastutil/longs/LongSet done
Event: 0.168 loading class it/unimi/dsi/fastutil/longs/LongCollection
Event: 0.168 loading class it/unimi/dsi/fastutil/longs/LongCollection done
Event: 0.168 loading class it/unimi/dsi/fastutil/longs/LongIterable
Event: 0.168 loading class it/unimi/dsi/fastutil/longs/LongIterable done
Event: 0.169 loading class it/unimi/dsi/fastutil/objects/ObjectSet
Event: 0.169 loading class it/unimi/dsi/fastutil/objects/ObjectSet done
Event: 0.169 loading class it/unimi/dsi/fastutil/objects/ObjectCollection
Event: 0.169 loading class it/unimi/dsi/fastutil/objects/ObjectCollection done
Event: 0.169 loading class it/unimi/dsi/fastutil/objects/ObjectIterable
Event: 0.169 loading class it/unimi/dsi/fastutil/objects/ObjectIterable done
Event: 0.170 loading class it/unimi/dsi/fastutil/longs/Long2DoubleMap$FastEntrySet
Event: 0.170 loading class it/unimi/dsi/fastutil/longs/Long2DoubleMap$FastEntrySet done
Event: 0.170 loading class java/util/concurrent/ThreadPoolExecutor
Event: 0.170 loading class java/util/concurrent/AbstractExecutorService
Event: 0.170 loading class java/util/concurrent/ExecutorService
Event: 0.171 loading class java/util/concurrent/Executor
Event: 0.171 loading class java/util/concurrent/Executor done
Event: 0.171 loading class java/util/concurrent/ExecutorService done
Event: 0.171 loading class java/util/concurrent/AbstractExecutorService done
Event: 0.171 loading class java/util/concurrent/ThreadPoolExecutor done
Event: 0.171 loading class java/util/concurrent/ThreadPoolExecutor$AbortPolicy
Event: 0.171 loading class java/util/concurrent/ThreadPoolExecutor$AbortPolicy done
Event: 0.171 loading class java/util/concurrent/TimeUnit
Event: 0.171 loading class java/util/concurrent/TimeUnit done
Event: 0.171 loading class java/util/concurrent/TimeUnit$1
Event: 0.171 loading class java/util/concurrent/TimeUnit$1 done
Event: 0.171 loading class java/util/concurrent/TimeUnit$2
Event: 0.171 loading class java/util/concurrent/TimeUnit$2 done
Event: 0.171 loading class java/util/concurrent/TimeUnit$3
Event: 0.171 loading class java/util/concurrent/TimeUnit$3 done
Event: 0.171 loading class java/util/concurrent/TimeUnit$4
Event: 0.171 loading class java/util/concurrent/TimeUnit$4 done
Event: 0.171 loading class java/util/concurrent/TimeUnit$5
Event: 0.171 loading class java/util/concurrent/TimeUnit$5 done
Event: 0.171 loading class java/util/concurrent/TimeUnit$6
Event: 0.171 loading class java/util/concurrent/TimeUnit$6 done
Event: 0.171 loading class java/util/concurrent/TimeUnit$7
Event: 0.171 loading class java/util/concurrent/TimeUnit$7 done
Event: 0.171 loading class java/util/concurrent/LinkedBlockingQueue
Event: 0.171 loading class java/util/AbstractQueue
Event: 0.171 loading class java/util/AbstractQueue done
Event: 0.171 loading class java/util/concurrent/LinkedBlockingQueue done
Event: 0.172 loading class java/util/concurrent/locks/ReentrantLock$NonfairSync
Event: 0.172 loading class java/util/concurrent/locks/ReentrantLock$Sync
Event: 0.172 loading class java/util/concurrent/locks/AbstractQueuedSynchronizer
Event: 0.172 loading class java/util/concurrent/locks/AbstractOwnableSynchronizer
Event: 0.172 loading class java/util/concurrent/locks/AbstractOwnableSynchronizer done
Event: 0.172 loading class java/util/concurrent/locks/AbstractQueuedSynchronizer done
Event: 0.172 loading class java/util/concurrent/locks/ReentrantLock$Sync done
Event: 0.172 loading class java/util/concurrent/locks/ReentrantLock$NonfairSync done
Event: 0.172 loading class java/util/concurrent/locks/AbstractQueuedSynchronizer$Node
Event: 0.172 loading class java/util/concurrent/locks/AbstractQueuedSynchronizer$Node done
Event: 0.172 loading class java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject
Event: 0.172 loading class java/util/concurrent/locks/Condition
Event: 0.172 loading class java/util/concurrent/locks/Condition done
Event: 0.172 loading class java/util/concurrent/locks/AbstractQueuedSynchronizer$ConditionObject done
Event: 0.172 loading class java/util/concurrent/LinkedBlockingQueue$Node
Event: 0.172 loading class java/util/concurrent/LinkedBlockingQueue$Node done
Event: 0.172 loading class java/util/concurrent/ThreadPoolExecutor$CallerRunsPolicy
Event: 0.172 loading class java/util/concurrent/ThreadPoolExecutor$CallerRunsPolicy done
Event: 0.172 loading class java/util/concurrent/Executors
Event: 0.172 loading class java/util/concurrent/Executors done
Event: 0.172 loading class java/util/concurrent/Executors$DefaultThreadFactory
Event: 0.172 loading class java/util/concurrent/ThreadFactory
Event: 0.172 loading class java/util/concurrent/ThreadFactory done
Event: 0.172 loading class java/util/concurrent/Executors$DefaultThreadFactory done
Event: 0.173 loading class it/unimi/dsi/fastutil/ints/IntOpenHashSet$SetIterator
Event: 0.173 loading class it/unimi/dsi/fastutil/ints/IntOpenHashSet$SetIterator done
Event: 0.173 loading class it/unimi/dsi/fastutil/ints/AbstractIntIterator
Event: 0.173 loading class it/unimi/dsi/fastutil/ints/AbstractIntIterator done
Event: 0.173 loading class java/lang/IllegalStateException
Event: 0.173 loading class java/lang/IllegalStateException done
Event: 0.173 loading class java/util/NoSuchElementException
Event: 0.173 loading class java/util/NoSuchElementException done
Event: 0.173 loading class it/unimi/dsi/fastutil/objects/ObjectIterators
Event: 0.173 loading class it/unimi/dsi/fastutil/objects/ObjectIterators done
Event: 0.174 loading class it/unimi/dsi/fastutil/objects/ObjectList
Event: 0.174 loading class it/unimi/dsi/fastutil/objects/ObjectList done
Event: 0.174 loading class it/unimi/dsi/fastutil/objects/ObjectIterator
Event: 0.174 loading class it/unimi/dsi/fastutil/objects/ObjectIterator done
Event: 0.174 loading class it/unimi/dsi/fastutil/objects/ObjectListIterator
Event: 0.174 loading class it/unimi/dsi/fastutil/objects/ObjectListIterator done
Event: 0.174 loading class it/unimi/dsi/fastutil/objects/ObjectBidirectionalIterator
Event: 0.174 loading class it/unimi/dsi/fastutil/objects/ObjectBidirectionalIterator done
Event: 0.175 loading class it/unimi/dsi/fastutil/BidirectionalIterator
Event: 0.175 loading class it/unimi/dsi/fastutil/BidirectionalIterator done
Event: 0.175 loading class it/unimi/dsi/fastutil/objects/ObjectIterators$EmptyIterator
Event: 0.175 loading class it/unimi/dsi/fastutil/objects/ObjectIterators$EmptyIterator done
Event: 0.175 loading class it/unimi/dsi/fastutil/objects/AbstractObjectListIterator
Event: 0.175 loading class it/unimi/dsi/fastutil/objects/AbstractObjectListIterator done
Event: 0.175 loading class it/unimi/dsi/fastutil/objects/AbstractObjectBidirectionalIterator
Event: 0.175 loading class it/unimi/dsi/fastutil/objects/AbstractObjectBidirectionalIterator done
Event: 0.175 loading class it/unimi/dsi/fastutil/objects/AbstractObjectIterator
Event: 0.175 loading class it/unimi/dsi/fastutil/objects/AbstractObjectIterator done
Event: 0.176 loading class edu/umd/marbl/mhap/sketch/ZeroNGramsFoundException
Event: 0.176 loading class edu/umd/marbl/mhap/sketch/ZeroNGramsFoundException done
Event: 0.176 loading class edu/umd/marbl/mhap/impl/FastaData
Event: 0.176 loading class edu/umd/marbl/mhap/impl/FastaData done
Event: 0.181 loading class java/util/concurrent/ConcurrentLinkedQueue
Event: 0.181 loading class java/util/concurrent/ConcurrentLinkedQueue done
Event: 0.181 loading class java/util/concurrent/ConcurrentLinkedQueue$Node
Event: 0.181 loading class java/util/concurrent/ConcurrentLinkedQueue$Node done
Event: 0.184 loading class edu/umd/marbl/mhap/impl/SequenceSketchStreamer$2
Event: 0.184 loading class edu/umd/marbl/mhap/impl/SequenceSketchStreamer$2 done
Event: 0.184 loading class java/util/concurrent/ThreadPoolExecutor$Worker
Event: 0.184 loading class java/util/concurrent/ThreadPoolExecutor$Worker done
Event: 0.185 Thread 0x00002ade4c499800 Thread added: 0x00002ade4c499800
Event: 0.185 Thread 0x00002ade4c49b000 Thread added: 0x00002ade4c49b000
Event: 0.185 Thread 0x00002ade4c49c800 Thread added: 0x00002ade4c49c800
Event: 0.185 Thread 0x00002ade4c49e800 Thread added: 0x00002ade4c49e800
Event: 0.185 Thread 0x00002ade4c4a0800 Thread added: 0x00002ade4c4a0800
Event: 0.185 Thread 0x00002ade4c4a2000 Thread added: 0x00002ade4c4a2000
Event: 0.185 Thread 0x00002ade4c4a4000 Thread added: 0x00002ade4c4a4000
Event: 0.186 Thread 0x00002ade4c4a6000 Thread added: 0x00002ade4c4a6000
Event: 0.186 Thread 0x00002ade4c4a7800 Thread added: 0x00002ade4c4a7800
Event: 0.186 Thread 0x00002ade4c4a9800 Thread added: 0x00002ade4c4a9800
Event: 0.186 Thread 0x00002ade4c4ab800 Thread added: 0x00002ade4c4ab800
Event: 0.186 Thread 0x00002ade4c4ad800 Thread added: 0x00002ade4c4ad800
Event: 0.187 Thread 0x00002ade4c4af000 Thread added: 0x00002ade4c4af000
Event: 0.188 Thread 0x00002ade4c4b1000 Thread added: 0x00002ade4c4b1000
Event: 0.188 Thread 0x00002ade4c4b3000 Thread added: 0x00002ade4c4b3000
Event: 0.189 Thread 0x00002ade4c4b5000 Thread added: 0x00002ade4c4b5000
Event: 0.189 loading class java/util/concurrent/locks/LockSupport
Event: 0.189 loading class edu/umd/marbl/mhap/utils/ReadBuffer
Event: 0.189 loading class edu/umd/marbl/mhap/utils/ReadBuffer done
Event: 0.189 loading class java/util/concurrent/locks/LockSupport done
Event: 0.190 loading class sun/nio/ch/Interruptible
Event: 0.190 loading class sun/nio/ch/Interruptible done

Dynamic libraries:
666600000-7c0080000 rw-p 00000000 00:00 0 
7c0080000-800000000 ---p 00000000 00:00 0 
2ade497f7000-2ade49819000 r-xp 00000000 08:01 1574237                    /usr/lib64/ld-2.17.so
2ade49819000-2ade4981a000 rw-p 00000000 00:00 0 
2ade4981a000-2ade4981b000 r--p 00000000 00:00 0 
2ade4981b000-2ade4981c000 rw-p 00000000 00:00 0 
2ade4981c000-2ade49824000 rw-s 00000000 08:01 131257                     /tmp/hsperfdata_uqkweigh/30362
2ade49824000-2ade49825000 rw-p 00000000 00:00 0 
2ade49825000-2ade4982c000 r--s 000f5000 08:01 3439046                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/jfr.jar
2ade4982c000-2ade49832000 rw-p 00000000 00:00 0 
2ade49832000-2ade49836000 ---p 00000000 00:00 0 
2ade49836000-2ade49a18000 rw-p 00000000 00:00 0 
2ade49a18000-2ade49a19000 r--p 00021000 08:01 1574237                    /usr/lib64/ld-2.17.so
2ade49a19000-2ade49a1a000 rw-p 00022000 08:01 1574237                    /usr/lib64/ld-2.17.so
2ade49a1a000-2ade49a1b000 rw-p 00000000 00:00 0 
2ade49a1b000-2ade49a32000 r-xp 00000000 08:01 1580265                    /usr/lib64/libpthread-2.17.so
2ade49a32000-2ade49c31000 ---p 00017000 08:01 1580265                    /usr/lib64/libpthread-2.17.so
2ade49c31000-2ade49c32000 r--p 00016000 08:01 1580265                    /usr/lib64/libpthread-2.17.so
2ade49c32000-2ade49c33000 rw-p 00017000 08:01 1580265                    /usr/lib64/libpthread-2.17.so
2ade49c33000-2ade49c37000 rw-p 00000000 00:00 0 
2ade49c37000-2ade49c4c000 r-xp 00000000 08:01 1580533                    /usr/lib64/libz.so.1.2.7
2ade49c4c000-2ade49e4b000 ---p 00015000 08:01 1580533                    /usr/lib64/libz.so.1.2.7
2ade49e4b000-2ade49e4c000 r--p 00014000 08:01 1580533                    /usr/lib64/libz.so.1.2.7
2ade49e4c000-2ade49e4d000 rw-p 00015000 08:01 1580533                    /usr/lib64/libz.so.1.2.7
2ade49e4d000-2ade49e5b000 r-xp 00000000 08:01 3670765                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/jli/libjli.so
2ade49e5b000-2ade4a05a000 ---p 0000e000 08:01 3670765                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/jli/libjli.so
2ade4a05a000-2ade4a05b000 r--p 0000d000 08:01 3670765                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/jli/libjli.so
2ade4a05b000-2ade4a05c000 rw-p 0000e000 08:01 3670765                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/jli/libjli.so
2ade4a05c000-2ade4a05e000 r-xp 00000000 08:01 1584815                    /usr/lib64/libdl-2.17.so
2ade4a05e000-2ade4a25e000 ---p 00002000 08:01 1584815                    /usr/lib64/libdl-2.17.so
2ade4a25e000-2ade4a25f000 r--p 00002000 08:01 1584815                    /usr/lib64/libdl-2.17.so
2ade4a25f000-2ade4a260000 rw-p 00003000 08:01 1584815                    /usr/lib64/libdl-2.17.so
2ade4a260000-2ade4a424000 r-xp 00000000 08:01 1580239                    /usr/lib64/libc-2.17.so
2ade4a424000-2ade4a623000 ---p 001c4000 08:01 1580239                    /usr/lib64/libc-2.17.so
2ade4a623000-2ade4a627000 r--p 001c3000 08:01 1580239                    /usr/lib64/libc-2.17.so
2ade4a627000-2ade4a629000 rw-p 001c7000 08:01 1580239                    /usr/lib64/libc-2.17.so
2ade4a629000-2ade4a62e000 rw-p 00000000 00:00 0 
2ade4a62e000-2ade4b3ca000 r-xp 00000000 08:01 3670800                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/server/libjvm.so
2ade4b3ca000-2ade4b5ca000 ---p 00d9c000 08:01 3670800                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/server/libjvm.so
2ade4b5ca000-2ade4b677000 r--p 00d9c000 08:01 3670800                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/server/libjvm.so
2ade4b677000-2ade4b6a2000 rw-p 00e49000 08:01 3670800                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/server/libjvm.so
2ade4b6a2000-2ade4b6d8000 rw-p 00000000 00:00 0 
2ade4b6d8000-2ade4b7c1000 r-xp 00000000 08:01 1604939                    /usr/lib64/libstdc++.so.6.0.19
2ade4b7c1000-2ade4b9c1000 ---p 000e9000 08:01 1604939                    /usr/lib64/libstdc++.so.6.0.19
2ade4b9c1000-2ade4b9c9000 r--p 000e9000 08:01 1604939                    /usr/lib64/libstdc++.so.6.0.19
2ade4b9c9000-2ade4b9cb000 rw-p 000f1000 08:01 1604939                    /usr/lib64/libstdc++.so.6.0.19
2ade4b9cb000-2ade4b9e0000 rw-p 00000000 00:00 0 
2ade4b9e0000-2ade4bae1000 r-xp 00000000 08:01 1584817                    /usr/lib64/libm-2.17.so
2ade4bae1000-2ade4bce0000 ---p 00101000 08:01 1584817                    /usr/lib64/libm-2.17.so
2ade4bce0000-2ade4bce1000 r--p 00100000 08:01 1584817                    /usr/lib64/libm-2.17.so
2ade4bce1000-2ade4bce2000 rw-p 00101000 08:01 1584817                    /usr/lib64/libm-2.17.so
2ade4bce2000-2ade4bcf7000 r-xp 00000000 08:01 1587514                    /usr/lib64/libgcc_s-4.8.5-20150702.so.1
2ade4bcf7000-2ade4bef6000 ---p 00015000 08:01 1587514                    /usr/lib64/libgcc_s-4.8.5-20150702.so.1
2ade4bef6000-2ade4bef7000 r--p 00014000 08:01 1587514                    /usr/lib64/libgcc_s-4.8.5-20150702.so.1
2ade4bef7000-2ade4bef8000 rw-p 00015000 08:01 1587514                    /usr/lib64/libgcc_s-4.8.5-20150702.so.1
2ade4bef8000-2ade4bef9000 ---p 00000000 00:00 0 
2ade4bef9000-2ade4c000000 rw-p 00000000 00:00 0 
2ade4c000000-2ade4c749000 rw-p 00000000 00:00 0 
2ade4c749000-2ade50000000 ---p 00000000 00:00 0 
2ade50000000-2ade50007000 r-xp 00000000 08:01 1584831                    /usr/lib64/librt-2.17.so
2ade50007000-2ade50206000 ---p 00007000 08:01 1584831                    /usr/lib64/librt-2.17.so
2ade50206000-2ade50207000 r--p 00006000 08:01 1584831                    /usr/lib64/librt-2.17.so
2ade50207000-2ade50208000 rw-p 00007000 08:01 1584831                    /usr/lib64/librt-2.17.so
2ade50208000-2ade50215000 r-xp 00000000 08:01 3670795                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libverify.so
2ade50215000-2ade50414000 ---p 0000d000 08:01 3670795                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libverify.so
2ade50414000-2ade50416000 r--p 0000c000 08:01 3670795                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libverify.so
2ade50416000-2ade50417000 rw-p 0000e000 08:01 3670795                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libverify.so
2ade50417000-2ade50441000 r-xp 00000000 08:01 3670778                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libjava.so
2ade50441000-2ade50640000 ---p 0002a000 08:01 3670778                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libjava.so
2ade50640000-2ade50641000 r--p 00029000 08:01 3670778                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libjava.so
2ade50641000-2ade50643000 rw-p 0002a000 08:01 3670778                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libjava.so
2ade50643000-2ade5064f000 r-xp 00000000 08:01 1584824                    /usr/lib64/libnss_files-2.17.so
2ade5064f000-2ade5084e000 ---p 0000c000 08:01 1584824                    /usr/lib64/libnss_files-2.17.so
2ade5084e000-2ade5084f000 r--p 0000b000 08:01 1584824                    /usr/lib64/libnss_files-2.17.so
2ade5084f000-2ade50850000 rw-p 0000c000 08:01 1584824                    /usr/lib64/libnss_files-2.17.so
2ade50850000-2ade50856000 rw-p 00000000 00:00 0 
2ade50856000-2ade5085e000 r-xp 00000000 08:01 3670796                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libzip.so
2ade5085e000-2ade50a5d000 ---p 00008000 08:01 3670796                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libzip.so
2ade50a5d000-2ade50a5e000 r--p 00007000 08:01 3670796                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libzip.so
2ade50a5e000-2ade50a5f000 rw-p 00008000 08:01 3670796                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libzip.so
2ade50a5f000-2ade50d5f000 rwxp 00000000 00:00 0 
2ade50d5f000-2ade5fa5f000 ---p 00000000 00:00 0 
2ade5fa5f000-2ade5fa6b000 rw-p 00000000 00:00 0 
2ade5fa6b000-2ade5fe1f000 ---p 00000000 00:00 0 
2ade5fe1f000-2ade61021000 rw-p 00000000 00:00 0 
2ade61021000-2ade61022000 ---p 00000000 00:00 0 
2ade61022000-2ade61122000 rw-p 00000000 00:00 0 
2ade61122000-2ade61123000 ---p 00000000 00:00 0 
2ade61123000-2ade61223000 rw-p 00000000 00:00 0 
2ade61223000-2ade61224000 ---p 00000000 00:00 0 
2ade61224000-2ade61324000 rw-p 00000000 00:00 0 
2ade61324000-2ade61325000 ---p 00000000 00:00 0 
2ade61325000-2ade61425000 rw-p 00000000 00:00 0 
2ade61425000-2ade61426000 ---p 00000000 00:00 0 
2ade61426000-2ade61526000 rw-p 00000000 00:00 0 
2ade61526000-2ade61527000 ---p 00000000 00:00 0 
2ade61527000-2ade61627000 rw-p 00000000 00:00 0 
2ade61627000-2ade61628000 ---p 00000000 00:00 0 
2ade61628000-2ade61728000 rw-p 00000000 00:00 0 
2ade61728000-2ade61729000 ---p 00000000 00:00 0 
2ade61729000-2ade61829000 rw-p 00000000 00:00 0 
2ade61829000-2ade6182a000 ---p 00000000 00:00 0 
2ade6182a000-2ade6192a000 rw-p 00000000 00:00 0 
2ade6192a000-2ade6192b000 ---p 00000000 00:00 0 
2ade6192b000-2ade61a2b000 rw-p 00000000 00:00 0 
2ade61a2b000-2ade61a2c000 ---p 00000000 00:00 0 
2ade61a2c000-2ade61b2c000 rw-p 00000000 00:00 0 
2ade61b2c000-2ade61b2d000 ---p 00000000 00:00 0 
2ade61b2d000-2ade61c2d000 rw-p 00000000 00:00 0 
2ade61c2d000-2ade61c2e000 ---p 00000000 00:00 0 
2ade61c2e000-2ade61d2e000 rw-p 00000000 00:00 0 
2ade61d2e000-2ade61d2f000 ---p 00000000 00:00 0 
2ade61d2f000-2ade61e2f000 rw-p 00000000 00:00 0 
2ade61e2f000-2ade61e30000 ---p 00000000 00:00 0 
2ade61e30000-2ade62e7d000 rw-p 00000000 00:00 0 
2ade62e7d000-2ade631fd000 ---p 00000000 00:00 0 
2ade631fd000-2ade632ce000 rw-p 00000000 00:00 0 
2ade632ce000-2ade6349f000 r--s 04495000 08:01 3439053                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/rt.jar
2ade6349f000-2ade64000000 rw-p 00000000 00:00 0 
2ade64000000-2ade64021000 rw-p 00000000 00:00 0 
2ade64021000-2ade68000000 ---p 00000000 00:00 0 
2ade68000000-2ade68021000 rw-p 00000000 00:00 0 
2ade68021000-2ade6c000000 ---p 00000000 00:00 0 
2ade6c000000-2ade6c021000 rw-p 00000000 00:00 0 
2ade6c021000-2ade70000000 ---p 00000000 00:00 0 
2ade70000000-2ade70021000 rw-p 00000000 00:00 0 
2ade70021000-2ade74000000 ---p 00000000 00:00 0 
2ade74000000-2ade74021000 rw-p 00000000 00:00 0 
2ade74021000-2ade78000000 ---p 00000000 00:00 0 
2ade78000000-2ade78021000 rw-p 00000000 00:00 0 
2ade78021000-2ade7c000000 ---p 00000000 00:00 0 
2ade7c000000-2ade7c021000 rw-p 00000000 00:00 0 
2ade7c021000-2ade80000000 ---p 00000000 00:00 0 
2ade80000000-2ade80021000 rw-p 00000000 00:00 0 
2ade80021000-2ade84000000 ---p 00000000 00:00 0 
2ade84000000-2ade84021000 rw-p 00000000 00:00 0 
2ade84021000-2ade88000000 ---p 00000000 00:00 0 
2ade88000000-2ade88021000 rw-p 00000000 00:00 0 
2ade88021000-2ade8c000000 ---p 00000000 00:00 0 
2ade8c000000-2ade8c021000 rw-p 00000000 00:00 0 
2ade8c021000-2ade90000000 ---p 00000000 00:00 0 
2ade90000000-2ade90021000 rw-p 00000000 00:00 0 
2ade90021000-2ade94000000 ---p 00000000 00:00 0 
2ade94000000-2ade94021000 rw-p 00000000 00:00 0 
2ade94021000-2ade98000000 ---p 00000000 00:00 0 
2ade98000000-2ade98021000 rw-p 00000000 00:00 0 
2ade98021000-2ade9c000000 ---p 00000000 00:00 0 
2ade9c000000-2ade9c021000 rw-p 00000000 00:00 0 
2ade9c021000-2adea0000000 ---p 00000000 00:00 0 
2adea0000000-2adea0021000 rw-p 00000000 00:00 0 
2adea0021000-2adea4000000 ---p 00000000 00:00 0 
2adea4000000-2adeb14f8000 rw-p 00000000 00:00 0 
2adeb14f8000-2adeb14f9000 ---p 00000000 00:00 0 
2adeb14f9000-2adeb15f9000 rw-p 00000000 00:00 0 
2adeb15f9000-2adeb15fc000 ---p 00000000 00:00 0 
2adeb15fc000-2adeb16fa000 rw-p 00000000 00:00 0 
2adeb16fa000-2adeb16fd000 ---p 00000000 00:00 0 
2adeb16fd000-2adeb17fb000 rw-p 00000000 00:00 0 
2adeb17fb000-2adeb17fe000 ---p 00000000 00:00 0 
2adeb17fe000-2adeb18fc000 rw-p 00000000 00:00 0 
2adeb18fc000-2adeb1900000 ---p 00000000 00:00 0 
2adeb1900000-2adeb19fd000 rw-p 00000000 00:00 0 
2adeb19fd000-2adeb1a01000 ---p 00000000 00:00 0 
2adeb1a01000-2adeb1afe000 rw-p 00000000 00:00 0 
2adeb1afe000-2adeb1b02000 ---p 00000000 00:00 0 
2adeb1b02000-2adeb1bff000 rw-p 00000000 00:00 0 
2adeb1bff000-2adeb1c03000 ---p 00000000 00:00 0 
2adeb1c03000-2adeb1d00000 rw-p 00000000 00:00 0 
2adeb1d00000-2adeb1d04000 ---p 00000000 00:00 0 
2adeb1d04000-2adeb1e01000 rw-p 00000000 00:00 0 
2adeb1e01000-2adeb1e05000 ---p 00000000 00:00 0 
2adeb1e05000-2adeb1f02000 rw-p 00000000 00:00 0 
2adeb1f02000-2adeb1f06000 ---p 00000000 00:00 0 
2adeb1f06000-2adeb2003000 rw-p 00000000 00:00 0 
2adeb2003000-2adeb2007000 ---p 00000000 00:00 0 
2adeb2007000-2adeb2124000 rw-p 00000000 00:00 0 
2adeb2124000-2adeb2128000 ---p 00000000 00:00 0 
2adeb2128000-2adeb2225000 rw-p 00000000 00:00 0 
2adeb2225000-2adeb2229000 ---p 00000000 00:00 0 
2adeb2229000-2adeb2326000 rw-p 00000000 00:00 0 
2adeb2326000-2adeb232a000 ---p 00000000 00:00 0 
2adeb232a000-2adeb2435000 rw-p 00000000 00:00 0 
2adeb2435000-2adeb2439000 ---p 00000000 00:00 0 
2adeb2439000-2adeb25ae000 rw-p 00000000 00:00 0 
2adeb25ae000-2adeb25b1000 ---p 00000000 00:00 0 
2adeb25b1000-2adeb26b0000 rw-p 00000000 00:00 0 
2adeb26b0000-2adeb26b1000 ---p 00000000 00:00 0 
2adeb26b1000-2adeb27e1000 rw-p 00000000 00:00 0 
2adeb27e1000-2adeb27e9000 rw-p 00000000 00:00 0 
2adeb27e9000-2adeb27f8000 r--s 000be000 00:2d 49157343                   /home/safs-data02/dennislab/sw/canu/2.1.1/share/java/classes/mhap-2.1.3.jar
2adeb27f8000-2adeb27f9000 rw-p 00000000 00:00 0 
2adeb27f9000-2adeb280b000 r-xp 00000000 08:01 3670789                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libnio.so
2adeb280b000-2adeb2a0a000 ---p 00012000 08:01 3670789                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libnio.so
2adeb2a0a000-2adeb2a0b000 r--p 00011000 08:01 3670789                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libnio.so
2adeb2a0b000-2adeb2a0c000 rw-p 00012000 08:01 3670789                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libnio.so
2adeb2a0c000-2adeb2a0d000 rw-p 00000000 00:00 0 
2adeb2a0d000-2adeb2a25000 r-xp 00000000 08:01 3670788                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libnet.so
2adeb2a25000-2adeb2c24000 ---p 00018000 08:01 3670788                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libnet.so
2adeb2c24000-2adeb2c25000 r--p 00017000 08:01 3670788                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libnet.so
2adeb2c25000-2adeb2c26000 rw-p 00018000 08:01 3670788                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/lib/amd64/libnet.so
2adeb2c26000-2adeb2d66000 rw-p 00000000 00:00 0 
2adeb2d66000-2adeb2d69000 ---p 00000000 00:00 0 
2adeb2d69000-2adeb2e6a000 rw-p 00000000 00:00 0 
2adeb2e6a000-2adeb2e6d000 ---p 00000000 00:00 0 
2adeb2e6d000-2adeb2f6c000 rw-p 00000000 00:00 0 
2adeb2f6c000-2adeb2f6f000 ---p 00000000 00:00 0 
2adeb2f6f000-2adeb306d000 rw-p 00000000 00:00 0 
2adeb306d000-2adeb3070000 ---p 00000000 00:00 0 
2adeb3070000-2adeb3170000 rw-p 00000000 00:00 0 
2adeb3170000-2adeb3173000 ---p 00000000 00:00 0 
2adeb3173000-2adeb3272000 rw-p 00000000 00:00 0 
2adeb3272000-2adeb3275000 ---p 00000000 00:00 0 
2adeb3275000-2adeb3374000 rw-p 00000000 00:00 0 
2adeb3374000-2adeb3377000 ---p 00000000 00:00 0 
2adeb3377000-2adeb3477000 rw-p 00000000 00:00 0 
2adeb3477000-2adeb347a000 ---p 00000000 00:00 0 
2adeb347a000-2adeb357a000 rw-p 00000000 00:00 0 
2adeb357a000-2adeb357d000 ---p 00000000 00:00 0 
2adeb357d000-2adeb367b000 rw-p 00000000 00:00 0 
2adeb367b000-2adeb367e000 ---p 00000000 00:00 0 
2adeb367e000-2adeb377c000 rw-p 00000000 00:00 0 
2adeb377c000-2adeb377f000 ---p 00000000 00:00 0 
2adeb377f000-2adeb3883000 rw-p 00000000 00:00 0 
2adeb3883000-2adeb3886000 ---p 00000000 00:00 0 
2adeb3886000-2adeb3984000 rw-p 00000000 00:00 0 
2adeb3984000-2adeb3987000 ---p 00000000 00:00 0 
2adeb3987000-2adeb3a87000 rw-p 00000000 00:00 0 
2adeb3a87000-2adeb3a8a000 ---p 00000000 00:00 0 
2adeb3a8a000-2adeb3b8d000 rw-p 00000000 00:00 0 
2adeb3b8d000-2adeb3b90000 ---p 00000000 00:00 0 
2adeb3b90000-2adeb3c90000 rw-p 00000000 00:00 0 
2adeb3c90000-2adeb3c93000 ---p 00000000 00:00 0 
2adeb3c93000-2adeb3daf000 rw-p 00000000 00:00 0 
2adeb4000000-2adeb4021000 rw-p 00000000 00:00 0 
2adeb4021000-2adeb8000000 ---p 00000000 00:00 0 
2adeb8000000-2adeb8021000 rw-p 00000000 00:00 0 
2adeb8021000-2adebc000000 ---p 00000000 00:00 0 
2adebc000000-2adebc021000 rw-p 00000000 00:00 0 
2adebc021000-2adec0000000 ---p 00000000 00:00 0 
5559683f3000-5559683f4000 r-xp 00000000 08:01 3670752                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/bin/java
5559685f3000-5559685f4000 r--p 00000000 08:01 3670752                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/bin/java
5559685f4000-5559685f5000 rw-p 00001000 08:01 3670752                    /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/bin/java
555969149000-55596916a000 rw-p 00000000 00:00 0                          [heap]
7ffc33727000-7ffc33749000 rw-p 00000000 00:00 0                          [stack]
7ffc33754000-7ffc33756000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

VM Arguments:
jvm_args: -XX:ParallelGCThreads=16 -Xms5530m -Xmx5530m 
java_command: /home/safs-data02/dennislab/sw/canu/2.1.1/bin/../share/java/classes/mhap-2.1.3.jar --repeat-weight 0.9 --repeat-idf-scale 10 -k 16 --store-full-id --num-hashes 256 --num-min-matches 3 --ordered-sketch-size 1000 --ordered-kmer-size 14 --threshold 0.8 --filter-threshold 0.0000001 --min-olap-length 500 --num-threads 16 -f ../../0-mercounts/P123.ms16.ignore.gz -p ./000205.input.fasta -q .
java_class_path (initial): /home/safs-data02/dennislab/sw/canu/2.1.1/bin/../share/java/classes/mhap-2.1.3.jar
Launcher Type: SUN_STANDARD

Environment Variables:
JAVA_HOME=/usr/java/latest
PATH=/safs-data02/dennislab/sw/canu/2.1.1/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/sdsc/bin:/opt/sdsc/sbin:/opt/bio/mpiblast/bin:/opt/bio/EMBOSS/bin:/opt/bio/clustalw/bin:/opt/bio/tcoffee/bin:/opt/bio/hmmer/bin:/opt/bio/phylip/exe:/opt/bio/mrbayes:/opt/bio/fasta:/opt/bio/glimmer/bin:/opt/bio/glimmer/scripts:/opt/bio/gmap/bin:/opt/bio/tigr/bin:/opt/bio/autodocksuite/bin:/opt/ganglia/bin:/opt/ganglia/sbin:/opt/pdsh/bin:/opt/puppetlabs/bin:/opt/rocks/bin:/opt/rocks/sbin:/safs-data02/dennislab/Public/Software/bin:/home/uqkweigh/bin
LD_LIBRARY_PATH=/opt/sdsc/lib
SHELL=/bin/bash
DISPLAY=localhost:10.0

Signal Handlers:
SIGSEGV: [libjvm.so+0xb66650], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGBUS: [libjvm.so+0xb66650], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGFPE: [libjvm.so+0x93c090], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGPIPE: [libjvm.so+0x93c090], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGXFSZ: [libjvm.so+0x93c090], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGILL: [libjvm.so+0x93c090], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGUSR1: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGUSR2: [libjvm.so+0x93bf40], sa_mask[0]=00000000000000000000000000000000, sa_flags=SA_RESTART|SA_SIGINFO
SIGHUP: [libjvm.so+0x93c150], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGINT: [libjvm.so+0x93c150], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGTERM: [libjvm.so+0x93c150], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGQUIT: [libjvm.so+0x93c150], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO

---------------  S Y S T E M  ---------------

OS:CentOS Linux release 7.9.2009 (Core)

uname:Linux 3.10.0-1160.36.2.el7.x86_64 #1 SMP Wed Jul 21 11:57:15 UTC 2021 x86_64
libc:glibc 2.17 NPTL 2.17 
rlimit: STACK infinity, CORE 0k, NPROC 4096, NOFILE 51200, AS 8650752k
load average:0.81 0.30 0.15

/proc/meminfo:
MemTotal:       264058724 kB
MemFree:        254840208 kB
MemAvailable:   260283252 kB
Buffers:           94512 kB
Cached:          6088584 kB
SwapCached:            0 kB
Active:          3869564 kB
Inactive:        2476704 kB
Active(anon):     163568 kB
Inactive(anon):    17960 kB
Active(file):    3705996 kB
Inactive(file):  2458744 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       4194300 kB
SwapFree:        4194300 kB
Dirty:               168 kB
Writeback:             0 kB
AnonPages:        103160 kB
Mapped:            46840 kB
Shmem:             18368 kB
Slab:             183220 kB
SReclaimable:      99572 kB
SUnreclaim:        83648 kB
KernelStack:        6288 kB
PageTables:         5164 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    136223660 kB
Committed_AS:    6324188 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      712336 kB
VmallocChunk:   34224736252 kB
Percpu:             6400 kB
HardwareCorrupted:     0 kB
AnonHugePages:     34816 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      127816 kB
DirectMap2M:     7116800 kB
DirectMap1G:    263192576 kB

container (cgroup) information:
container_type: cgroupv1
cpu_cpuset_cpus: failed
cpu_memory_nodes: failed
active_processor_count: 16
cpu_quota: -2
cpu_period: -2
cpu_shares: -2
memory_limit_in_bytes: -2
memory_and_swap_limit_in_bytes: -2
memory_soft_limit_in_bytes: -2
memory_usage_in_bytes: -2
memory_max_usage_in_bytes: -2

CPU:total 24 (initial active 16) (12 cores per cpu, 2 threads per core) family 6 model 63 stepping 2, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, lzcnt, ht, tsc, tscinvbit, bmi1, bmi2

/proc/cpuinfo:
processor   : 0
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1294.097
cache size  : 30720 KB
physical id : 0
siblings    : 12
core id     : 0
cpu cores   : 12
apicid      : 0
initial apicid  : 0
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 4999.95
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 1
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 3267.364
cache size  : 30720 KB
physical id : 1
siblings    : 12
core id     : 0
cpu cores   : 12
apicid      : 32
initial apicid  : 32
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 5004.10
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 2
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1599.884
cache size  : 30720 KB
physical id : 0
siblings    : 12
core id     : 1
cpu cores   : 12
apicid      : 2
initial apicid  : 2
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 4999.95
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 3
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1786.956
cache size  : 30720 KB
physical id : 1
siblings    : 12
core id     : 1
cpu cores   : 12
apicid      : 34
initial apicid  : 34
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 5004.10
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 4
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1295.776
cache size  : 30720 KB
physical id : 0
siblings    : 12
core id     : 2
cpu cores   : 12
apicid      : 4
initial apicid  : 4
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 4999.95
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 5
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1798.858
cache size  : 30720 KB
physical id : 1
siblings    : 12
core id     : 2
cpu cores   : 12
apicid      : 36
initial apicid  : 36
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 5004.10
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 6
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1269.378
cache size  : 30720 KB
physical id : 0
siblings    : 12
core id     : 3
cpu cores   : 12
apicid      : 6
initial apicid  : 6
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 4999.95
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 7
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1899.108
cache size  : 30720 KB
physical id : 1
siblings    : 12
core id     : 3
cpu cores   : 12
apicid      : 38
initial apicid  : 38
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 5004.10
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 8
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1399.993
cache size  : 30720 KB
physical id : 0
siblings    : 12
core id     : 4
cpu cores   : 12
apicid      : 8
initial apicid  : 8
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 4999.95
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 9
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1200.103
cache size  : 30720 KB
physical id : 1
siblings    : 12
core id     : 4
cpu cores   : 12
apicid      : 40
initial apicid  : 40
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 5004.10
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 10
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1200.103
cache size  : 30720 KB
physical id : 0
siblings    : 12
core id     : 5
cpu cores   : 12
apicid      : 10
initial apicid  : 10
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 4999.95
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 11
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1199.951
cache size  : 30720 KB
physical id : 1
siblings    : 12
core id     : 5
cpu cores   : 12
apicid      : 42
initial apicid  : 42
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 5004.10
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 12
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1261.444
cache size  : 30720 KB
physical id : 0
siblings    : 12
core id     : 8
cpu cores   : 12
apicid      : 16
initial apicid  : 16
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 4999.95
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 13
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1199.951
cache size  : 30720 KB
physical id : 1
siblings    : 12
core id     : 8
cpu cores   : 12
apicid      : 48
initial apicid  : 48
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 5004.10
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 14
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1200.103
cache size  : 30720 KB
physical id : 0
siblings    : 12
core id     : 9
cpu cores   : 12
apicid      : 18
initial apicid  : 18
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 4999.95
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 15
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1200.103
cache size  : 30720 KB
physical id : 1
siblings    : 12
core id     : 9
cpu cores   : 12
apicid      : 50
initial apicid  : 50
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 5004.10
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 16
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1200.103
cache size  : 30720 KB
physical id : 0
siblings    : 12
core id     : 10
cpu cores   : 12
apicid      : 20
initial apicid  : 20
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 4999.95
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 17
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1199.951
cache size  : 30720 KB
physical id : 1
siblings    : 12
core id     : 10
cpu cores   : 12
apicid      : 52
initial apicid  : 52
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 5004.10
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 18
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1200.103
cache size  : 30720 KB
physical id : 0
siblings    : 12
core id     : 11
cpu cores   : 12
apicid      : 22
initial apicid  : 22
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 4999.95
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 19
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1199.951
cache size  : 30720 KB
physical id : 1
siblings    : 12
core id     : 11
cpu cores   : 12
apicid      : 54
initial apicid  : 54
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 5004.10
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 20
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1199.951
cache size  : 30720 KB
physical id : 0
siblings    : 12
core id     : 12
cpu cores   : 12
apicid      : 24
initial apicid  : 24
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 4999.95
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 21
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1200.103
cache size  : 30720 KB
physical id : 1
siblings    : 12
core id     : 12
cpu cores   : 12
apicid      : 56
initial apicid  : 56
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 5004.10
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 22
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1200.103
cache size  : 30720 KB
physical id : 0
siblings    : 12
core id     : 13
cpu cores   : 12
apicid      : 26
initial apicid  : 26
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 4999.95
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

processor   : 23
vendor_id   : GenuineIntel
cpu family  : 6
model       : 63
model name  : Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz
stepping    : 2
microcode   : 0x46
cpu MHz     : 1199.951
cache size  : 30720 KB
physical id : 1
siblings    : 12
core id     : 13
cpu cores   : 12
apicid      : 58
initial apicid  : 58
fpu     : yes
fpu_exception   : yes
cpuid level : 15
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm epb invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc dtherm ida arat pln pts md_clear spec_ctrl intel_stibp flush_l1d
bogomips    : 5004.10
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

Memory: 4k page, physical 7864320k(254840208k free), swap 4194300k(4194300k free)

vm_info: OpenJDK 64-Bit Server VM (25.292-b10) for linux-amd64 JRE (1.8.0_292-b10), built on Apr 21 2021 14:24:48 by "mockbuild" with gcc 4.8.5 20150623 (Red Hat 4.8.5-44)

time: Wed Sep 29 08:12:22 2021
timezone: AEST
elapsed time: 0.204240 seconds (0d 0h 0m 0s)
skoren commented 2 years ago

It seems your JVM isn't working properly or at least not working properly within compute nodes on the cluster. Canu is only trying to use 6gb and 16 cores and the machine it landed on reports over 200gb free. Despite this, the jobs are all almost immediately failing when the JVM starts up. I don't think it's related to JAVA_OPTS, that's not a canu variable but is an environmental variable that can sometimes restrict the JVM but the logs indicate canu properly reserved about 6gb on the command line and no other options show up.

We've seen these types of memory errors from the JVM before, always from openjdk (see #2011, #1438) which are resolved by adjusting how the jobs request memory from the cluster. It seems openjdk configuration on a cluster is quite finicky and I'd recommend using the standard Sun/Oracle JVM if possible. Given your genome is also small you could run on a single reserved node (say one of the 16 cores/188gb nodes) and add useGrid=false.

kattweigh commented 2 years ago

Okay so I ran with 16 cores and 188Gb mem and added the useGrid=false flag and got another step closer. It completed the correction step but stopped before trimming and assembling. The only 'error' log I could find was correction/2-correction/P123.readsToCorrect.err which just kindly says 'Bye'.

This time it generated the following files in the main directory:

The following files are in the correction/ directory:

The following files are in the 2-correction/ directory:

Do you think this stopped due to an issue with the JVM still?

Thanks

skoren commented 2 years ago

No, the JVM is only used in the beginning. Did you job exit due to a time limit? What's the slurm report for the job Canu ran in and the log Canu wrote to screen while running (probably saved to a file from slurm)?

kattweigh commented 2 years ago

Ah yes. This time there is an error in the slurm file (see below). It seems as though the coverage is too low because either I've put an incorrect prediction for the genome size or the reads couldn't be corrected properly. I am trying to do a bacterial genome assembly with these long reads. Maybe I don't have enough data? 

-- canu 2.1.1
--
-- CITATIONS
--
-- For 'standard' assemblies of PacBio or Nanopore reads:
--   Koren S, Walenz BP, Berlin K, Miller JR, Phillippy AM.
--   Canu: scalable and accurate long-read assembly via adaptive k-mer weighting and repeat separation.
--   Genome Res. 2017 May;27(5):722-736.
--   http://doi.org/10.1101/gr.215087.116
-- 
-- Read and contig alignments during correction and consensus use:
--   Šošic M, Šikic M.
--   Edlib: a C/C ++ library for fast, exact sequence alignment using edit distance.
--   Bioinformatics. 2017 May 1;33(9):1394-1395.
--   http://doi.org/10.1093/bioinformatics/btw753
-- 
-- Overlaps are generated using:
--   Berlin K, et al.
--   Assembling large genomes with single-molecule sequencing and locality-sensitive hashing.
--   Nat Biotechnol. 2015 Jun;33(6):623-30.
--   http://doi.org/10.1038/nbt.3238
-- 
--   Myers EW, et al.
--   A Whole-Genome Assembly of Drosophila.
--   Science. 2000 Mar 24;287(5461):2196-204.
--   http://doi.org/10.1126/science.287.5461.2196
-- 
-- Corrected read consensus sequences are generated using an algorithm derived from FALCON-sense:
--   Chin CS, et al.
--   Phased diploid genome assembly with single-molecule real-time sequencing.
--   Nat Methods. 2016 Dec;13(12):1050-1054.
--   http://doi.org/10.1038/nmeth.4035
-- 
-- Contig consensus sequences are generated using an algorithm derived from pbdagcon:
--   Chin CS, et al.
--   Nonhybrid, finished microbial genome assemblies from long-read SMRT sequencing data.
--   Nat Methods. 2013 Jun;10(6):563-9
--   http://doi.org/10.1038/nmeth.2474
-- 
-- CONFIGURE CANU
--
-- Detected Java(TM) Runtime Environment '1.8.0_292' (from 'java') with -d64 support.
-- Detected gnuplot version '4.6 patchlevel 2   ' (from 'gnuplot') and image format 'png'.
-- Detected 24 CPUs and 252 gigabytes of memory.
-- Detected Slurm with 'sinfo' binary in /usr/bin/sinfo.
-- Grid engine and staging disabled per useGrid=false option.
--
--                                (tag)Concurrency
--                         (tag)Threads          |
--                (tag)Memory         |          |
--        (tag)             |         |          |       total usage      algorithm
--        -------  ----------  --------   --------  --------------------  -----------------------------
-- Local: meryl     12.000 GB   16 CPUs x   1 job     12.000 GB  16 CPUs  (k-mer counting)
-- Local: hap        8.000 GB   16 CPUs x   1 job      8.000 GB  16 CPUs  (read-to-haplotype assignment)
-- Local: cormhap    6.000 GB   16 CPUs x   1 job      6.000 GB  16 CPUs  (overlap detection with mhap)
-- Local: obtovl     4.000 GB   16 CPUs x   1 job      4.000 GB  16 CPUs  (overlap detection)
-- Local: utgovl     4.000 GB   16 CPUs x   1 job      4.000 GB  16 CPUs  (overlap detection)
-- Local: cor        8.000 GB   16 CPUs x   1 job      8.000 GB  16 CPUs  (read correction)
-- Local: ovb        4.000 GB   16 CPUs x   1 job      4.000 GB  16 CPUs  (overlap store bucketizer)
-- Local: ovs        8.000 GB   16 CPUs x   1 job      8.000 GB  16 CPUs  (overlap store sorting)
-- Local: red       16.000 GB   16 CPUs x   1 job     16.000 GB  16 CPUs  (read error detection)
-- Local: oea        8.000 GB   16 CPUs x   1 job      8.000 GB  16 CPUs  (overlap error adjustment)
-- Local: bat       16.000 GB   16 CPUs x   1 job     16.000 GB  16 CPUs  (contig construction with bogart)
-- Local: cns        -.--- GB   16 CPUs x   - jobs     -.--- GB   - CPUs  (consensus)
--
-- Found untrimmed raw Nanopore reads in the input files.
--
-- Generating assembly 'P123' in '/home/safs-data01/uqkweigh/long-read-assemblies/canu_P123':
--    - correct raw reads.
--    - trim corrected reads.
--    - assemble corrected and trimmed reads.
--
-- Parameters:
--
--  genomeSize        8000000
--
--  Overlap Generation Limits:
--    corOvlErrorRate 0.3200 ( 32.00%)
--    obtOvlErrorRate 0.1200 ( 12.00%)
--    utgOvlErrorRate 0.1200 ( 12.00%)
--
--  Overlap Processing Limits:
--    corErrorRate    0.5000 ( 50.00%)
--    obtErrorRate    0.1200 ( 12.00%)
--    utgErrorRate    0.1200 ( 12.00%)
--    cnsErrorRate    0.2000 ( 20.00%)
--
--
-- BEGIN CORRECTION
--
----------------------------------------
-- Starting command on Wed Sep 29 11:29:46 2021 with 3357.319 GB free disk space

    cd .
    ./P123.seqStore.sh \
    > ./P123.seqStore.err 2>&1

-- Finished on Wed Sep 29 11:34:33 2021 (287 seconds) with 3350.169 GB free disk space
----------------------------------------
----------------------------------------
-- Starting command on Wed Sep 29 11:34:33 2021 with 3350.169 GB free disk space

    cd .
    /home/safs-data02/dennislab/sw/canu/2.1.1/bin/sqStoreDumpMetaData \
      -S ./P123.seqStore \
      -raw \
      -histogram \
      -lengths \
    > ./P123.seqStore/readlengths-cor.dat \
    2> ./P123.seqStore/readlengths-cor.err 

-- Finished on Wed Sep 29 11:34:33 2021 (lickety-split) with 3350.167 GB free disk space
----------------------------------------
----------------------------------------
-- Starting command on Wed Sep 29 11:34:34 2021 with 3350.167 GB free disk space

    cd .
    /home/safs-data02/dennislab/sw/canu/2.1.1/bin/sqStoreDumpMetaData \
      -S ./P123.seqStore \
      -raw \
      -histogram \
    > ./P123.seqStore/readlengths-cor.txt \
    2> ./P123.seqStore/readlengths-cor.err 

-- Finished on Wed Sep 29 11:34:34 2021 (fast as lightning) with 3350.167 GB free disk space
----------------------------------------
--
-- In sequence store './P123.seqStore':
--   Found 273588 reads.
--   Found 1600002219 bases (200 times coverage).
--    
--    G=1600002219                       sum of  ||               length     num
--    NG         length     index       lengths  ||                range    seqs
--    ----- ------------ --------- ------------  ||  ------------------- -------
--    00010        19251      6952    160006170  ||       1000-2784       104293|---------------------------------------------------------------
--    00020        15692     16213    320003241  ||       2785-4569        45018|----------------------------
--    00030        13253     27344    480011578  ||       4570-6354        30431|-------------------
--    00040        11273     40451    640001242  ||       6355-8139        23644|---------------
--    00050         9461     55942    800004794  ||       8140-9924        18572|------------
--    00060         7782     74569    960002912  ||       9925-11709       14474|---------
--    00070         6104     97726   1120006279  ||      11710-13494       11146|-------
--    00080         4361    128558   1280005941  ||      13495-15279        8240|-----
--    00090         2605    175511   1440004281  ||      15280-17064        5844|----
--    00100         1000    273587   1600002219  ||      17065-18849        4256|---
--    001.000x              273588   1600002219  ||      18850-20634        2805|--
--                                               ||      20635-22419        1840|--
--                                               ||      22420-24204        1181|-
--                                               ||      24205-25989         721|-
--                                               ||      25990-27774         447|-
--                                               ||      27775-29559         277|-
--                                               ||      29560-31344         158|-
--                                               ||      31345-33129          78|-
--                                               ||      33130-34914          45|-
--                                               ||      34915-36699          37|-
--                                               ||      36700-38484          20|-
--                                               ||      38485-40269          22|-
--                                               ||      40270-42054           8|-
--                                               ||      42055-43839          12|-
--                                               ||      43840-45624           5|-
--                                               ||      45625-47409           6|-
--                                               ||      47410-49194           2|-
--                                               ||      49195-50979           1|-
--                                               ||      50980-52764           1|-
--                                               ||      52765-54549           0|
--                                               ||      54550-56334           0|
--                                               ||      56335-58119           0|
--                                               ||      58120-59904           1|-
--                                               ||      59905-61689           1|-
--                                               ||      61690-63474           0|
--                                               ||      63475-65259           1|-
--                                               ||      65260-67044           0|
--                                               ||      67045-68829           0|
--                                               ||      68830-70614           0|
--                                               ||      70615-72399           0|
--                                               ||      72400-74184           0|
--                                               ||      74185-75969           0|
--                                               ||      75970-77754           0|
--                                               ||      77755-79539           0|
--                                               ||      79540-81324           0|
--                                               ||      81325-83109           0|
--                                               ||      83110-84894           0|
--                                               ||      84895-86679           0|
--                                               ||      86680-88464           0|
--                                               ||      88465-90249           1|-
--    
----------------------------------------
-- Starting command on Wed Sep 29 11:34:34 2021 with 3350.167 GB free disk space

    cd correction/0-mercounts
    ./meryl-configure.sh \
    > ./meryl-configure.err 2>&1

-- Finished on Wed Sep 29 11:34:35 2021 (one second) with 3350.167 GB free disk space
----------------------------------------
--  segments   memory batches
--  -------- -------- -------
--        01  3.76 GB       1
--        02  2.07 GB       1
--        04  1.07 GB       1
--        06  0.69 GB       1
--        08  0.57 GB       1
--        12  0.38 GB       1
--        16  0.32 GB       1
--
--  For 273588 reads with 1600002219 bases, limit to 16 batches.
--  Will count kmers using 01 jobs, each using 5 GB and 16 threads.
--
-- Finished stage 'merylConfigure', reset canuIteration.
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'meryl' concurrent execution on Wed Sep 29 11:34:35 2021 with 3350.167 GB free disk space (1 processes; 1 concurrently)

    cd correction/0-mercounts
    ./meryl-count.sh 1 > ./meryl-count.000001.out 2>&1

-- Finished on Wed Sep 29 11:35:32 2021 (57 seconds) with 3346.916 GB free disk space
----------------------------------------
-- Found 1 Kmer counting (meryl) outputs.
-- Finished stage 'cor-merylCountCheck', reset canuIteration.
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'meryl' concurrent execution on Wed Sep 29 11:35:32 2021 with 3346.916 GB free disk space (1 processes; 1 concurrently)

    cd correction/0-mercounts
    ./meryl-process.sh 1 > ./meryl-process.000001.out 2>&1

-- Finished on Wed Sep 29 11:35:42 2021 (10 seconds) with 3348.825 GB free disk space
----------------------------------------
-- Meryl finished successfully.  Kmer frequency histogram:
--
--  16-mers                                                                                           Fraction
--    Occurrences   NumMers                                                                         Unique Total
--       1-     1         0                                                                        0.0000 0.0000
--       2-     2 157678176 ********************************************************************** 0.5087 0.2783
--       3-     4  95407920 ******************************************                             0.7159 0.4484
--       5-     7  34827076 ***************                                                        0.8723 0.6348
--       8-    11  12711850 *****                                                                  0.9443 0.7675
--      12-    16   5064643 **                                                                     0.9747 0.8520
--      17-    22   2192387                                                                        0.9879 0.9048
--      23-    29   1014241                                                                        0.9939 0.9378
--      30-    37    499922                                                                        0.9968 0.9588
--      38-    46    255911                                                                        0.9983 0.9723
--      47-    56    136484                                                                        0.9990 0.9811
--      57-    67     75580                                                                        0.9994 0.9870
--      68-    79     43182                                                                        0.9997 0.9909
--      80-    92     24642                                                                        0.9998 0.9936
--      93-   106     14454                                                                        0.9999 0.9953
--     107-   121      8649                                                                        0.9999 0.9966
--     122-   137      5309                                                                        1.0000 0.9974
--     138-   154      3356                                                                        1.0000 0.9980
--     155-   172      2061                                                                        1.0000 0.9984
--     173-   191      1274                                                                        1.0000 0.9987
--     192-   211       800                                                                        1.0000 0.9989
--     212-   232       506                                                                        1.0000 0.9990
--     233-   254       392                                                                        1.0000 0.9991
--     255-   277       244                                                                        1.0000 0.9992
--     278-   301       184                                                                        1.0000 0.9993
--     302-   326       107                                                                        1.0000 0.9993
--     327-   352        80                                                                        1.0000 0.9993
--     353-   379        69                                                                        1.0000 0.9994
--     380-   407        61                                                                        1.0000 0.9994
--     408-   436        54                                                                        1.0000 0.9994
--     437-   466        38                                                                        1.0000 0.9994
--     467-   497        20                                                                        1.0000 0.9995
--     498-   529        25                                                                        1.0000 0.9995
--     530-   562        32                                                                        1.0000 0.9995
--     563-   596        25                                                                        1.0000 0.9995
--     597-   631        16                                                                        1.0000 0.9995
--     632-   667         7                                                                        1.0000 0.9995
--     668-   704        12                                                                        1.0000 0.9995
--     705-   742         4                                                                        1.0000 0.9995
--     743-   781        12                                                                        1.0000 0.9995
--     782-   821         5                                                                        1.0000 0.9995
--
--           0 (max occurrences)
--  1133087206 (total mers, non-unique)
--   309969971 (distinct mers, non-unique)
--           0 (unique mers)
-- Finished stage 'meryl-process', reset canuIteration.
--
-- Removing meryl database 'correction/0-mercounts/P123.ms16'.
--
-- OVERLAPPER (mhap) (correction)
--
-- Set corMhapSensitivity=low based on read coverage of 200.
--
-- PARAMETERS: hashes=256, minMatches=3, threshold=0.8
--
-- Given 5.4 GB, can fit 16200 reads per block.
-- For 206 blocks, set stride to 51 blocks.
-- Logging partitioning to 'correction/1-overlapper/partitioning.log'.
-- Configured 205 mhap precompute jobs.
-- Configured 511 mhap overlap jobs.
-- Finished stage 'cor-mhapConfigure', reset canuIteration.
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'cormhap' concurrent execution on Wed Sep 29 11:35:47 2021 with 3350.161 GB free disk space (205 processes; 1 concurrently)

    cd correction/1-overlapper
    ./precompute.sh 1 > ./precompute.000001.out 2>&1
    ./precompute.sh 2 > ./precompute.000002.out 2>&1
    ./precompute.sh 3 > ./precompute.000003.out 2>&1
    ./precompute.sh 4 > ./precompute.000004.out 2>&1
    ./precompute.sh 5 > ./precompute.000005.out 2>&1
    ./precompute.sh 6 > ./precompute.000006.out 2>&1
    ./precompute.sh 7 > ./precompute.000007.out 2>&1
    ./precompute.sh 8 > ./precompute.000008.out 2>&1
    ./precompute.sh 9 > ./precompute.000009.out 2>&1
    ./precompute.sh 10 > ./precompute.000010.out 2>&1
    ./precompute.sh 11 > ./precompute.000011.out 2>&1
    ./precompute.sh 12 > ./precompute.000012.out 2>&1
    ./precompute.sh 13 > ./precompute.000013.out 2>&1
    ./precompute.sh 14 > ./precompute.000014.out 2>&1
    ./precompute.sh 15 > ./precompute.000015.out 2>&1
    ./precompute.sh 16 > ./precompute.000016.out 2>&1
    ./precompute.sh 17 > ./precompute.000017.out 2>&1
    ./precompute.sh 18 > ./precompute.000018.out 2>&1
    ./precompute.sh 19 > ./precompute.000019.out 2>&1
    ./precompute.sh 20 > ./precompute.000020.out 2>&1
    ./precompute.sh 21 > ./precompute.000021.out 2>&1
    ./precompute.sh 22 > ./precompute.000022.out 2>&1
    ./precompute.sh 23 > ./precompute.000023.out 2>&1
    ./precompute.sh 24 > ./precompute.000024.out 2>&1
    ./precompute.sh 25 > ./precompute.000025.out 2>&1
    ./precompute.sh 26 > ./precompute.000026.out 2>&1
    ./precompute.sh 27 > ./precompute.000027.out 2>&1
    ./precompute.sh 28 > ./precompute.000028.out 2>&1
    ./precompute.sh 29 > ./precompute.000029.out 2>&1
    ./precompute.sh 30 > ./precompute.000030.out 2>&1
    ./precompute.sh 31 > ./precompute.000031.out 2>&1
    ./precompute.sh 32 > ./precompute.000032.out 2>&1
    ./precompute.sh 33 > ./precompute.000033.out 2>&1
    ./precompute.sh 34 > ./precompute.000034.out 2>&1
    ./precompute.sh 35 > ./precompute.000035.out 2>&1
    ./precompute.sh 36 > ./precompute.000036.out 2>&1
    ./precompute.sh 37 > ./precompute.000037.out 2>&1
    ./precompute.sh 38 > ./precompute.000038.out 2>&1
    ./precompute.sh 39 > ./precompute.000039.out 2>&1
    ./precompute.sh 40 > ./precompute.000040.out 2>&1
    ./precompute.sh 41 > ./precompute.000041.out 2>&1
    ./precompute.sh 42 > ./precompute.000042.out 2>&1
    ./precompute.sh 43 > ./precompute.000043.out 2>&1
    ./precompute.sh 44 > ./precompute.000044.out 2>&1
    ./precompute.sh 45 > ./precompute.000045.out 2>&1
    ./precompute.sh 46 > ./precompute.000046.out 2>&1
    ./precompute.sh 47 > ./precompute.000047.out 2>&1
    ./precompute.sh 48 > ./precompute.000048.out 2>&1
    ./precompute.sh 49 > ./precompute.000049.out 2>&1
    ./precompute.sh 50 > ./precompute.000050.out 2>&1
    ./precompute.sh 51 > ./precompute.000051.out 2>&1
    ./precompute.sh 52 > ./precompute.000052.out 2>&1
    ./precompute.sh 53 > ./precompute.000053.out 2>&1
    ./precompute.sh 54 > ./precompute.000054.out 2>&1
    ./precompute.sh 55 > ./precompute.000055.out 2>&1
    ./precompute.sh 56 > ./precompute.000056.out 2>&1
    ./precompute.sh 57 > ./precompute.000057.out 2>&1
    ./precompute.sh 58 > ./precompute.000058.out 2>&1
    ./precompute.sh 59 > ./precompute.000059.out 2>&1
    ./precompute.sh 60 > ./precompute.000060.out 2>&1
    ./precompute.sh 61 > ./precompute.000061.out 2>&1
    ./precompute.sh 62 > ./precompute.000062.out 2>&1
    ./precompute.sh 63 > ./precompute.000063.out 2>&1
    ./precompute.sh 64 > ./precompute.000064.out 2>&1
    ./precompute.sh 65 > ./precompute.000065.out 2>&1
    ./precompute.sh 66 > ./precompute.000066.out 2>&1
    ./precompute.sh 67 > ./precompute.000067.out 2>&1
    ./precompute.sh 68 > ./precompute.000068.out 2>&1
    ./precompute.sh 69 > ./precompute.000069.out 2>&1
    ./precompute.sh 70 > ./precompute.000070.out 2>&1
    ./precompute.sh 71 > ./precompute.000071.out 2>&1
    ./precompute.sh 72 > ./precompute.000072.out 2>&1
    ./precompute.sh 73 > ./precompute.000073.out 2>&1
    ./precompute.sh 74 > ./precompute.000074.out 2>&1
    ./precompute.sh 75 > ./precompute.000075.out 2>&1
    ./precompute.sh 76 > ./precompute.000076.out 2>&1
    ./precompute.sh 77 > ./precompute.000077.out 2>&1
    ./precompute.sh 78 > ./precompute.000078.out 2>&1
    ./precompute.sh 79 > ./precompute.000079.out 2>&1
    ./precompute.sh 80 > ./precompute.000080.out 2>&1
    ./precompute.sh 81 > ./precompute.000081.out 2>&1
    ./precompute.sh 82 > ./precompute.000082.out 2>&1
    ./precompute.sh 83 > ./precompute.000083.out 2>&1
    ./precompute.sh 84 > ./precompute.000084.out 2>&1
    ./precompute.sh 85 > ./precompute.000085.out 2>&1
    ./precompute.sh 86 > ./precompute.000086.out 2>&1
    ./precompute.sh 87 > ./precompute.000087.out 2>&1
    ./precompute.sh 88 > ./precompute.000088.out 2>&1
    ./precompute.sh 89 > ./precompute.000089.out 2>&1
    ./precompute.sh 90 > ./precompute.000090.out 2>&1
    ./precompute.sh 91 > ./precompute.000091.out 2>&1
    ./precompute.sh 92 > ./precompute.000092.out 2>&1
    ./precompute.sh 93 > ./precompute.000093.out 2>&1
    ./precompute.sh 94 > ./precompute.000094.out 2>&1
    ./precompute.sh 95 > ./precompute.000095.out 2>&1
    ./precompute.sh 96 > ./precompute.000096.out 2>&1
    ./precompute.sh 97 > ./precompute.000097.out 2>&1
    ./precompute.sh 98 > ./precompute.000098.out 2>&1
    ./precompute.sh 99 > ./precompute.000099.out 2>&1
    ./precompute.sh 100 > ./precompute.000100.out 2>&1
    ./precompute.sh 101 > ./precompute.000101.out 2>&1
    ./precompute.sh 102 > ./precompute.000102.out 2>&1
    ./precompute.sh 103 > ./precompute.000103.out 2>&1
    ./precompute.sh 104 > ./precompute.000104.out 2>&1
    ./precompute.sh 105 > ./precompute.000105.out 2>&1
    ./precompute.sh 106 > ./precompute.000106.out 2>&1
    ./precompute.sh 107 > ./precompute.000107.out 2>&1
    ./precompute.sh 108 > ./precompute.000108.out 2>&1
    ./precompute.sh 109 > ./precompute.000109.out 2>&1
    ./precompute.sh 110 > ./precompute.000110.out 2>&1
    ./precompute.sh 111 > ./precompute.000111.out 2>&1
    ./precompute.sh 112 > ./precompute.000112.out 2>&1
    ./precompute.sh 113 > ./precompute.000113.out 2>&1
    ./precompute.sh 114 > ./precompute.000114.out 2>&1
    ./precompute.sh 115 > ./precompute.000115.out 2>&1
    ./precompute.sh 116 > ./precompute.000116.out 2>&1
    ./precompute.sh 117 > ./precompute.000117.out 2>&1
    ./precompute.sh 118 > ./precompute.000118.out 2>&1
    ./precompute.sh 119 > ./precompute.000119.out 2>&1
    ./precompute.sh 120 > ./precompute.000120.out 2>&1
    ./precompute.sh 121 > ./precompute.000121.out 2>&1
    ./precompute.sh 122 > ./precompute.000122.out 2>&1
    ./precompute.sh 123 > ./precompute.000123.out 2>&1
    ./precompute.sh 124 > ./precompute.000124.out 2>&1
    ./precompute.sh 125 > ./precompute.000125.out 2>&1
    ./precompute.sh 126 > ./precompute.000126.out 2>&1
    ./precompute.sh 127 > ./precompute.000127.out 2>&1
    ./precompute.sh 128 > ./precompute.000128.out 2>&1
    ./precompute.sh 129 > ./precompute.000129.out 2>&1
    ./precompute.sh 130 > ./precompute.000130.out 2>&1
    ./precompute.sh 131 > ./precompute.000131.out 2>&1
    ./precompute.sh 132 > ./precompute.000132.out 2>&1
    ./precompute.sh 133 > ./precompute.000133.out 2>&1
    ./precompute.sh 134 > ./precompute.000134.out 2>&1
    ./precompute.sh 135 > ./precompute.000135.out 2>&1
    ./precompute.sh 136 > ./precompute.000136.out 2>&1
    ./precompute.sh 137 > ./precompute.000137.out 2>&1
    ./precompute.sh 138 > ./precompute.000138.out 2>&1
    ./precompute.sh 139 > ./precompute.000139.out 2>&1
    ./precompute.sh 140 > ./precompute.000140.out 2>&1
    ./precompute.sh 141 > ./precompute.000141.out 2>&1
    ./precompute.sh 142 > ./precompute.000142.out 2>&1
    ./precompute.sh 143 > ./precompute.000143.out 2>&1
    ./precompute.sh 144 > ./precompute.000144.out 2>&1
    ./precompute.sh 145 > ./precompute.000145.out 2>&1
    ./precompute.sh 146 > ./precompute.000146.out 2>&1
    ./precompute.sh 147 > ./precompute.000147.out 2>&1
    ./precompute.sh 148 > ./precompute.000148.out 2>&1
    ./precompute.sh 149 > ./precompute.000149.out 2>&1
    ./precompute.sh 150 > ./precompute.000150.out 2>&1
    ./precompute.sh 151 > ./precompute.000151.out 2>&1
    ./precompute.sh 152 > ./precompute.000152.out 2>&1
    ./precompute.sh 153 > ./precompute.000153.out 2>&1
    ./precompute.sh 154 > ./precompute.000154.out 2>&1
    ./precompute.sh 155 > ./precompute.000155.out 2>&1
    ./precompute.sh 156 > ./precompute.000156.out 2>&1
    ./precompute.sh 157 > ./precompute.000157.out 2>&1
    ./precompute.sh 158 > ./precompute.000158.out 2>&1
    ./precompute.sh 159 > ./precompute.000159.out 2>&1
    ./precompute.sh 160 > ./precompute.000160.out 2>&1
    ./precompute.sh 161 > ./precompute.000161.out 2>&1
    ./precompute.sh 162 > ./precompute.000162.out 2>&1
    ./precompute.sh 163 > ./precompute.000163.out 2>&1
    ./precompute.sh 164 > ./precompute.000164.out 2>&1
    ./precompute.sh 165 > ./precompute.000165.out 2>&1
    ./precompute.sh 166 > ./precompute.000166.out 2>&1
    ./precompute.sh 167 > ./precompute.000167.out 2>&1
    ./precompute.sh 168 > ./precompute.000168.out 2>&1
    ./precompute.sh 169 > ./precompute.000169.out 2>&1
    ./precompute.sh 170 > ./precompute.000170.out 2>&1
    ./precompute.sh 171 > ./precompute.000171.out 2>&1
    ./precompute.sh 172 > ./precompute.000172.out 2>&1
    ./precompute.sh 173 > ./precompute.000173.out 2>&1
    ./precompute.sh 174 > ./precompute.000174.out 2>&1
    ./precompute.sh 175 > ./precompute.000175.out 2>&1
    ./precompute.sh 176 > ./precompute.000176.out 2>&1
    ./precompute.sh 177 > ./precompute.000177.out 2>&1
    ./precompute.sh 178 > ./precompute.000178.out 2>&1
    ./precompute.sh 179 > ./precompute.000179.out 2>&1
    ./precompute.sh 180 > ./precompute.000180.out 2>&1
    ./precompute.sh 181 > ./precompute.000181.out 2>&1
    ./precompute.sh 182 > ./precompute.000182.out 2>&1
    ./precompute.sh 183 > ./precompute.000183.out 2>&1
    ./precompute.sh 184 > ./precompute.000184.out 2>&1
    ./precompute.sh 185 > ./precompute.000185.out 2>&1
    ./precompute.sh 186 > ./precompute.000186.out 2>&1
    ./precompute.sh 187 > ./precompute.000187.out 2>&1
    ./precompute.sh 188 > ./precompute.000188.out 2>&1
    ./precompute.sh 189 > ./precompute.000189.out 2>&1
    ./precompute.sh 190 > ./precompute.000190.out 2>&1
    ./precompute.sh 191 > ./precompute.000191.out 2>&1
    ./precompute.sh 192 > ./precompute.000192.out 2>&1
    ./precompute.sh 193 > ./precompute.000193.out 2>&1
    ./precompute.sh 194 > ./precompute.000194.out 2>&1
    ./precompute.sh 195 > ./precompute.000195.out 2>&1
    ./precompute.sh 196 > ./precompute.000196.out 2>&1
    ./precompute.sh 197 > ./precompute.000197.out 2>&1
    ./precompute.sh 198 > ./precompute.000198.out 2>&1
    ./precompute.sh 199 > ./precompute.000199.out 2>&1
    ./precompute.sh 200 > ./precompute.000200.out 2>&1
    ./precompute.sh 201 > ./precompute.000201.out 2>&1
    ./precompute.sh 202 > ./precompute.000202.out 2>&1
    ./precompute.sh 203 > ./precompute.000203.out 2>&1
    ./precompute.sh 204 > ./precompute.000204.out 2>&1
    ./precompute.sh 205 > ./precompute.000205.out 2>&1

-- Finished on Wed Sep 29 12:02:27 2021 (1600 seconds) with 3345.537 GB free disk space
----------------------------------------
-- All 205 mhap precompute jobs finished successfully.
-- Finished stage 'cor-mhapPrecomputeCheck', reset canuIteration.
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'cormhap' concurrent execution on Wed Sep 29 12:02:27 2021 with 3345.537 GB free disk space (511 processes; 1 concurrently)

    cd correction/1-overlapper
    ./mhap.sh 1 > ./mhap.000001.out 2>&1
    ./mhap.sh 2 > ./mhap.000002.out 2>&1
    ./mhap.sh 3 > ./mhap.000003.out 2>&1
    ./mhap.sh 4 > ./mhap.000004.out 2>&1
    ./mhap.sh 5 > ./mhap.000005.out 2>&1
    ./mhap.sh 6 > ./mhap.000006.out 2>&1
    ./mhap.sh 7 > ./mhap.000007.out 2>&1
    ./mhap.sh 8 > ./mhap.000008.out 2>&1
    ./mhap.sh 9 > ./mhap.000009.out 2>&1
    ./mhap.sh 10 > ./mhap.000010.out 2>&1
    ./mhap.sh 11 > ./mhap.000011.out 2>&1
    ./mhap.sh 12 > ./mhap.000012.out 2>&1
    ./mhap.sh 13 > ./mhap.000013.out 2>&1
    ./mhap.sh 14 > ./mhap.000014.out 2>&1
    ./mhap.sh 15 > ./mhap.000015.out 2>&1
    ./mhap.sh 16 > ./mhap.000016.out 2>&1
    ./mhap.sh 17 > ./mhap.000017.out 2>&1
    ./mhap.sh 18 > ./mhap.000018.out 2>&1
    ./mhap.sh 19 > ./mhap.000019.out 2>&1
    ./mhap.sh 20 > ./mhap.000020.out 2>&1
    ./mhap.sh 21 > ./mhap.000021.out 2>&1
    ./mhap.sh 22 > ./mhap.000022.out 2>&1
    ./mhap.sh 23 > ./mhap.000023.out 2>&1
    ./mhap.sh 24 > ./mhap.000024.out 2>&1
    ./mhap.sh 25 > ./mhap.000025.out 2>&1
    ./mhap.sh 26 > ./mhap.000026.out 2>&1
    ./mhap.sh 27 > ./mhap.000027.out 2>&1
    ./mhap.sh 28 > ./mhap.000028.out 2>&1
    ./mhap.sh 29 > ./mhap.000029.out 2>&1
    ./mhap.sh 30 > ./mhap.000030.out 2>&1
    ./mhap.sh 31 > ./mhap.000031.out 2>&1
    ./mhap.sh 32 > ./mhap.000032.out 2>&1
    ./mhap.sh 33 > ./mhap.000033.out 2>&1
    ./mhap.sh 34 > ./mhap.000034.out 2>&1
    ./mhap.sh 35 > ./mhap.000035.out 2>&1
    ./mhap.sh 36 > ./mhap.000036.out 2>&1
    ./mhap.sh 37 > ./mhap.000037.out 2>&1
    ./mhap.sh 38 > ./mhap.000038.out 2>&1
    ./mhap.sh 39 > ./mhap.000039.out 2>&1
    ./mhap.sh 40 > ./mhap.000040.out 2>&1
    ./mhap.sh 41 > ./mhap.000041.out 2>&1
    ./mhap.sh 42 > ./mhap.000042.out 2>&1
    ./mhap.sh 43 > ./mhap.000043.out 2>&1
    ./mhap.sh 44 > ./mhap.000044.out 2>&1
    ./mhap.sh 45 > ./mhap.000045.out 2>&1
    ./mhap.sh 46 > ./mhap.000046.out 2>&1
    ./mhap.sh 47 > ./mhap.000047.out 2>&1
    ./mhap.sh 48 > ./mhap.000048.out 2>&1
    ./mhap.sh 49 > ./mhap.000049.out 2>&1
    ./mhap.sh 50 > ./mhap.000050.out 2>&1
    ./mhap.sh 51 > ./mhap.000051.out 2>&1
    ./mhap.sh 52 > ./mhap.000052.out 2>&1
    ./mhap.sh 53 > ./mhap.000053.out 2>&1
    ./mhap.sh 54 > ./mhap.000054.out 2>&1
    ./mhap.sh 55 > ./mhap.000055.out 2>&1
    ./mhap.sh 56 > ./mhap.000056.out 2>&1
    ./mhap.sh 57 > ./mhap.000057.out 2>&1
    ./mhap.sh 58 > ./mhap.000058.out 2>&1
    ./mhap.sh 59 > ./mhap.000059.out 2>&1
    ./mhap.sh 60 > ./mhap.000060.out 2>&1
    ./mhap.sh 61 > ./mhap.000061.out 2>&1
    ./mhap.sh 62 > ./mhap.000062.out 2>&1
    ./mhap.sh 63 > ./mhap.000063.out 2>&1
    ./mhap.sh 64 > ./mhap.000064.out 2>&1
    ./mhap.sh 65 > ./mhap.000065.out 2>&1
    ./mhap.sh 66 > ./mhap.000066.out 2>&1
    ./mhap.sh 67 > ./mhap.000067.out 2>&1
    ./mhap.sh 68 > ./mhap.000068.out 2>&1
    ./mhap.sh 69 > ./mhap.000069.out 2>&1
    ./mhap.sh 70 > ./mhap.000070.out 2>&1
    ./mhap.sh 71 > ./mhap.000071.out 2>&1
    ./mhap.sh 72 > ./mhap.000072.out 2>&1
    ./mhap.sh 73 > ./mhap.000073.out 2>&1
    ./mhap.sh 74 > ./mhap.000074.out 2>&1
    ./mhap.sh 75 > ./mhap.000075.out 2>&1
    ./mhap.sh 76 > ./mhap.000076.out 2>&1
    ./mhap.sh 77 > ./mhap.000077.out 2>&1
    ./mhap.sh 78 > ./mhap.000078.out 2>&1
    ./mhap.sh 79 > ./mhap.000079.out 2>&1
    ./mhap.sh 80 > ./mhap.000080.out 2>&1
    ./mhap.sh 81 > ./mhap.000081.out 2>&1
    ./mhap.sh 82 > ./mhap.000082.out 2>&1
    ./mhap.sh 83 > ./mhap.000083.out 2>&1
    ./mhap.sh 84 > ./mhap.000084.out 2>&1
    ./mhap.sh 85 > ./mhap.000085.out 2>&1
    ./mhap.sh 86 > ./mhap.000086.out 2>&1
    ./mhap.sh 87 > ./mhap.000087.out 2>&1
    ./mhap.sh 88 > ./mhap.000088.out 2>&1
    ./mhap.sh 89 > ./mhap.000089.out 2>&1
    ./mhap.sh 90 > ./mhap.000090.out 2>&1
    ./mhap.sh 91 > ./mhap.000091.out 2>&1
    ./mhap.sh 92 > ./mhap.000092.out 2>&1
    ./mhap.sh 93 > ./mhap.000093.out 2>&1
    ./mhap.sh 94 > ./mhap.000094.out 2>&1
    ./mhap.sh 95 > ./mhap.000095.out 2>&1
    ./mhap.sh 96 > ./mhap.000096.out 2>&1
    ./mhap.sh 97 > ./mhap.000097.out 2>&1
    ./mhap.sh 98 > ./mhap.000098.out 2>&1
    ./mhap.sh 99 > ./mhap.000099.out 2>&1
    ./mhap.sh 100 > ./mhap.000100.out 2>&1
    ./mhap.sh 101 > ./mhap.000101.out 2>&1
    ./mhap.sh 102 > ./mhap.000102.out 2>&1
    ./mhap.sh 103 > ./mhap.000103.out 2>&1
    ./mhap.sh 104 > ./mhap.000104.out 2>&1
    ./mhap.sh 105 > ./mhap.000105.out 2>&1
    ./mhap.sh 106 > ./mhap.000106.out 2>&1
    ./mhap.sh 107 > ./mhap.000107.out 2>&1
    ./mhap.sh 108 > ./mhap.000108.out 2>&1
    ./mhap.sh 109 > ./mhap.000109.out 2>&1
    ./mhap.sh 110 > ./mhap.000110.out 2>&1
    ./mhap.sh 111 > ./mhap.000111.out 2>&1
    ./mhap.sh 112 > ./mhap.000112.out 2>&1
    ./mhap.sh 113 > ./mhap.000113.out 2>&1
    ./mhap.sh 114 > ./mhap.000114.out 2>&1
    ./mhap.sh 115 > ./mhap.000115.out 2>&1
    ./mhap.sh 116 > ./mhap.000116.out 2>&1
    ./mhap.sh 117 > ./mhap.000117.out 2>&1
    ./mhap.sh 118 > ./mhap.000118.out 2>&1
    ./mhap.sh 119 > ./mhap.000119.out 2>&1
    ./mhap.sh 120 > ./mhap.000120.out 2>&1
    ./mhap.sh 121 > ./mhap.000121.out 2>&1
    ./mhap.sh 122 > ./mhap.000122.out 2>&1
    ./mhap.sh 123 > ./mhap.000123.out 2>&1
    ./mhap.sh 124 > ./mhap.000124.out 2>&1
    ./mhap.sh 125 > ./mhap.000125.out 2>&1
    ./mhap.sh 126 > ./mhap.000126.out 2>&1
    ./mhap.sh 127 > ./mhap.000127.out 2>&1
    ./mhap.sh 128 > ./mhap.000128.out 2>&1
    ./mhap.sh 129 > ./mhap.000129.out 2>&1
    ./mhap.sh 130 > ./mhap.000130.out 2>&1
    ./mhap.sh 131 > ./mhap.000131.out 2>&1
    ./mhap.sh 132 > ./mhap.000132.out 2>&1
    ./mhap.sh 133 > ./mhap.000133.out 2>&1
    ./mhap.sh 134 > ./mhap.000134.out 2>&1
    ./mhap.sh 135 > ./mhap.000135.out 2>&1
    ./mhap.sh 136 > ./mhap.000136.out 2>&1
    ./mhap.sh 137 > ./mhap.000137.out 2>&1
    ./mhap.sh 138 > ./mhap.000138.out 2>&1
    ./mhap.sh 139 > ./mhap.000139.out 2>&1
    ./mhap.sh 140 > ./mhap.000140.out 2>&1
    ./mhap.sh 141 > ./mhap.000141.out 2>&1
    ./mhap.sh 142 > ./mhap.000142.out 2>&1
    ./mhap.sh 143 > ./mhap.000143.out 2>&1
    ./mhap.sh 144 > ./mhap.000144.out 2>&1
    ./mhap.sh 145 > ./mhap.000145.out 2>&1
    ./mhap.sh 146 > ./mhap.000146.out 2>&1
    ./mhap.sh 147 > ./mhap.000147.out 2>&1
    ./mhap.sh 148 > ./mhap.000148.out 2>&1
    ./mhap.sh 149 > ./mhap.000149.out 2>&1
    ./mhap.sh 150 > ./mhap.000150.out 2>&1
    ./mhap.sh 151 > ./mhap.000151.out 2>&1
    ./mhap.sh 152 > ./mhap.000152.out 2>&1
    ./mhap.sh 153 > ./mhap.000153.out 2>&1
    ./mhap.sh 154 > ./mhap.000154.out 2>&1
    ./mhap.sh 155 > ./mhap.000155.out 2>&1
    ./mhap.sh 156 > ./mhap.000156.out 2>&1
    ./mhap.sh 157 > ./mhap.000157.out 2>&1
    ./mhap.sh 158 > ./mhap.000158.out 2>&1
    ./mhap.sh 159 > ./mhap.000159.out 2>&1
    ./mhap.sh 160 > ./mhap.000160.out 2>&1
    ./mhap.sh 161 > ./mhap.000161.out 2>&1
    ./mhap.sh 162 > ./mhap.000162.out 2>&1
    ./mhap.sh 163 > ./mhap.000163.out 2>&1
    ./mhap.sh 164 > ./mhap.000164.out 2>&1
    ./mhap.sh 165 > ./mhap.000165.out 2>&1
    ./mhap.sh 166 > ./mhap.000166.out 2>&1
    ./mhap.sh 167 > ./mhap.000167.out 2>&1
    ./mhap.sh 168 > ./mhap.000168.out 2>&1
    ./mhap.sh 169 > ./mhap.000169.out 2>&1
    ./mhap.sh 170 > ./mhap.000170.out 2>&1
    ./mhap.sh 171 > ./mhap.000171.out 2>&1
    ./mhap.sh 172 > ./mhap.000172.out 2>&1
    ./mhap.sh 173 > ./mhap.000173.out 2>&1
    ./mhap.sh 174 > ./mhap.000174.out 2>&1
    ./mhap.sh 175 > ./mhap.000175.out 2>&1
    ./mhap.sh 176 > ./mhap.000176.out 2>&1
    ./mhap.sh 177 > ./mhap.000177.out 2>&1
    ./mhap.sh 178 > ./mhap.000178.out 2>&1
    ./mhap.sh 179 > ./mhap.000179.out 2>&1
    ./mhap.sh 180 > ./mhap.000180.out 2>&1
    ./mhap.sh 181 > ./mhap.000181.out 2>&1
    ./mhap.sh 182 > ./mhap.000182.out 2>&1
    ./mhap.sh 183 > ./mhap.000183.out 2>&1
    ./mhap.sh 184 > ./mhap.000184.out 2>&1
    ./mhap.sh 185 > ./mhap.000185.out 2>&1
    ./mhap.sh 186 > ./mhap.000186.out 2>&1
    ./mhap.sh 187 > ./mhap.000187.out 2>&1
    ./mhap.sh 188 > ./mhap.000188.out 2>&1
    ./mhap.sh 189 > ./mhap.000189.out 2>&1
    ./mhap.sh 190 > ./mhap.000190.out 2>&1
    ./mhap.sh 191 > ./mhap.000191.out 2>&1
    ./mhap.sh 192 > ./mhap.000192.out 2>&1
    ./mhap.sh 193 > ./mhap.000193.out 2>&1
    ./mhap.sh 194 > ./mhap.000194.out 2>&1
    ./mhap.sh 195 > ./mhap.000195.out 2>&1
    ./mhap.sh 196 > ./mhap.000196.out 2>&1
    ./mhap.sh 197 > ./mhap.000197.out 2>&1
    ./mhap.sh 198 > ./mhap.000198.out 2>&1
    ./mhap.sh 199 > ./mhap.000199.out 2>&1
    ./mhap.sh 200 > ./mhap.000200.out 2>&1
    ./mhap.sh 201 > ./mhap.000201.out 2>&1
    ./mhap.sh 202 > ./mhap.000202.out 2>&1
    ./mhap.sh 203 > ./mhap.000203.out 2>&1
    ./mhap.sh 204 > ./mhap.000204.out 2>&1
    ./mhap.sh 205 > ./mhap.000205.out 2>&1
    ./mhap.sh 206 > ./mhap.000206.out 2>&1
    ./mhap.sh 207 > ./mhap.000207.out 2>&1
    ./mhap.sh 208 > ./mhap.000208.out 2>&1
    ./mhap.sh 209 > ./mhap.000209.out 2>&1
    ./mhap.sh 210 > ./mhap.000210.out 2>&1
    ./mhap.sh 211 > ./mhap.000211.out 2>&1
    ./mhap.sh 212 > ./mhap.000212.out 2>&1
    ./mhap.sh 213 > ./mhap.000213.out 2>&1
    ./mhap.sh 214 > ./mhap.000214.out 2>&1
    ./mhap.sh 215 > ./mhap.000215.out 2>&1
    ./mhap.sh 216 > ./mhap.000216.out 2>&1
    ./mhap.sh 217 > ./mhap.000217.out 2>&1
    ./mhap.sh 218 > ./mhap.000218.out 2>&1
    ./mhap.sh 219 > ./mhap.000219.out 2>&1
    ./mhap.sh 220 > ./mhap.000220.out 2>&1
    ./mhap.sh 221 > ./mhap.000221.out 2>&1
    ./mhap.sh 222 > ./mhap.000222.out 2>&1
    ./mhap.sh 223 > ./mhap.000223.out 2>&1
    ./mhap.sh 224 > ./mhap.000224.out 2>&1
    ./mhap.sh 225 > ./mhap.000225.out 2>&1
    ./mhap.sh 226 > ./mhap.000226.out 2>&1
    ./mhap.sh 227 > ./mhap.000227.out 2>&1
    ./mhap.sh 228 > ./mhap.000228.out 2>&1
    ./mhap.sh 229 > ./mhap.000229.out 2>&1
    ./mhap.sh 230 > ./mhap.000230.out 2>&1
    ./mhap.sh 231 > ./mhap.000231.out 2>&1
    ./mhap.sh 232 > ./mhap.000232.out 2>&1
    ./mhap.sh 233 > ./mhap.000233.out 2>&1
    ./mhap.sh 234 > ./mhap.000234.out 2>&1
    ./mhap.sh 235 > ./mhap.000235.out 2>&1
    ./mhap.sh 236 > ./mhap.000236.out 2>&1
    ./mhap.sh 237 > ./mhap.000237.out 2>&1
    ./mhap.sh 238 > ./mhap.000238.out 2>&1
    ./mhap.sh 239 > ./mhap.000239.out 2>&1
    ./mhap.sh 240 > ./mhap.000240.out 2>&1
    ./mhap.sh 241 > ./mhap.000241.out 2>&1
    ./mhap.sh 242 > ./mhap.000242.out 2>&1
    ./mhap.sh 243 > ./mhap.000243.out 2>&1
    ./mhap.sh 244 > ./mhap.000244.out 2>&1
    ./mhap.sh 245 > ./mhap.000245.out 2>&1
    ./mhap.sh 246 > ./mhap.000246.out 2>&1
    ./mhap.sh 247 > ./mhap.000247.out 2>&1
    ./mhap.sh 248 > ./mhap.000248.out 2>&1
    ./mhap.sh 249 > ./mhap.000249.out 2>&1
    ./mhap.sh 250 > ./mhap.000250.out 2>&1
    ./mhap.sh 251 > ./mhap.000251.out 2>&1
    ./mhap.sh 252 > ./mhap.000252.out 2>&1
    ./mhap.sh 253 > ./mhap.000253.out 2>&1
    ./mhap.sh 254 > ./mhap.000254.out 2>&1
    ./mhap.sh 255 > ./mhap.000255.out 2>&1
    ./mhap.sh 256 > ./mhap.000256.out 2>&1
    ./mhap.sh 257 > ./mhap.000257.out 2>&1
    ./mhap.sh 258 > ./mhap.000258.out 2>&1
    ./mhap.sh 259 > ./mhap.000259.out 2>&1
    ./mhap.sh 260 > ./mhap.000260.out 2>&1
    ./mhap.sh 261 > ./mhap.000261.out 2>&1
    ./mhap.sh 262 > ./mhap.000262.out 2>&1
    ./mhap.sh 263 > ./mhap.000263.out 2>&1
    ./mhap.sh 264 > ./mhap.000264.out 2>&1
    ./mhap.sh 265 > ./mhap.000265.out 2>&1
    ./mhap.sh 266 > ./mhap.000266.out 2>&1
    ./mhap.sh 267 > ./mhap.000267.out 2>&1
    ./mhap.sh 268 > ./mhap.000268.out 2>&1
    ./mhap.sh 269 > ./mhap.000269.out 2>&1
    ./mhap.sh 270 > ./mhap.000270.out 2>&1
    ./mhap.sh 271 > ./mhap.000271.out 2>&1
    ./mhap.sh 272 > ./mhap.000272.out 2>&1
    ./mhap.sh 273 > ./mhap.000273.out 2>&1
    ./mhap.sh 274 > ./mhap.000274.out 2>&1
    ./mhap.sh 275 > ./mhap.000275.out 2>&1
    ./mhap.sh 276 > ./mhap.000276.out 2>&1
    ./mhap.sh 277 > ./mhap.000277.out 2>&1
    ./mhap.sh 278 > ./mhap.000278.out 2>&1
    ./mhap.sh 279 > ./mhap.000279.out 2>&1
    ./mhap.sh 280 > ./mhap.000280.out 2>&1
    ./mhap.sh 281 > ./mhap.000281.out 2>&1
    ./mhap.sh 282 > ./mhap.000282.out 2>&1
    ./mhap.sh 283 > ./mhap.000283.out 2>&1
    ./mhap.sh 284 > ./mhap.000284.out 2>&1
    ./mhap.sh 285 > ./mhap.000285.out 2>&1
    ./mhap.sh 286 > ./mhap.000286.out 2>&1
    ./mhap.sh 287 > ./mhap.000287.out 2>&1
    ./mhap.sh 288 > ./mhap.000288.out 2>&1
    ./mhap.sh 289 > ./mhap.000289.out 2>&1
    ./mhap.sh 290 > ./mhap.000290.out 2>&1
    ./mhap.sh 291 > ./mhap.000291.out 2>&1
    ./mhap.sh 292 > ./mhap.000292.out 2>&1
    ./mhap.sh 293 > ./mhap.000293.out 2>&1
    ./mhap.sh 294 > ./mhap.000294.out 2>&1
    ./mhap.sh 295 > ./mhap.000295.out 2>&1
    ./mhap.sh 296 > ./mhap.000296.out 2>&1
    ./mhap.sh 297 > ./mhap.000297.out 2>&1
    ./mhap.sh 298 > ./mhap.000298.out 2>&1
    ./mhap.sh 299 > ./mhap.000299.out 2>&1
    ./mhap.sh 300 > ./mhap.000300.out 2>&1
    ./mhap.sh 301 > ./mhap.000301.out 2>&1
    ./mhap.sh 302 > ./mhap.000302.out 2>&1
    ./mhap.sh 303 > ./mhap.000303.out 2>&1
    ./mhap.sh 304 > ./mhap.000304.out 2>&1
    ./mhap.sh 305 > ./mhap.000305.out 2>&1
    ./mhap.sh 306 > ./mhap.000306.out 2>&1
    ./mhap.sh 307 > ./mhap.000307.out 2>&1
    ./mhap.sh 308 > ./mhap.000308.out 2>&1
    ./mhap.sh 309 > ./mhap.000309.out 2>&1
    ./mhap.sh 310 > ./mhap.000310.out 2>&1
    ./mhap.sh 311 > ./mhap.000311.out 2>&1
    ./mhap.sh 312 > ./mhap.000312.out 2>&1
    ./mhap.sh 313 > ./mhap.000313.out 2>&1
    ./mhap.sh 314 > ./mhap.000314.out 2>&1
    ./mhap.sh 315 > ./mhap.000315.out 2>&1
    ./mhap.sh 316 > ./mhap.000316.out 2>&1
    ./mhap.sh 317 > ./mhap.000317.out 2>&1
    ./mhap.sh 318 > ./mhap.000318.out 2>&1
    ./mhap.sh 319 > ./mhap.000319.out 2>&1
    ./mhap.sh 320 > ./mhap.000320.out 2>&1
    ./mhap.sh 321 > ./mhap.000321.out 2>&1
    ./mhap.sh 322 > ./mhap.000322.out 2>&1
    ./mhap.sh 323 > ./mhap.000323.out 2>&1
    ./mhap.sh 324 > ./mhap.000324.out 2>&1
    ./mhap.sh 325 > ./mhap.000325.out 2>&1
    ./mhap.sh 326 > ./mhap.000326.out 2>&1
    ./mhap.sh 327 > ./mhap.000327.out 2>&1
    ./mhap.sh 328 > ./mhap.000328.out 2>&1
    ./mhap.sh 329 > ./mhap.000329.out 2>&1
    ./mhap.sh 330 > ./mhap.000330.out 2>&1
    ./mhap.sh 331 > ./mhap.000331.out 2>&1
    ./mhap.sh 332 > ./mhap.000332.out 2>&1
    ./mhap.sh 333 > ./mhap.000333.out 2>&1
    ./mhap.sh 334 > ./mhap.000334.out 2>&1
    ./mhap.sh 335 > ./mhap.000335.out 2>&1
    ./mhap.sh 336 > ./mhap.000336.out 2>&1
    ./mhap.sh 337 > ./mhap.000337.out 2>&1
    ./mhap.sh 338 > ./mhap.000338.out 2>&1
    ./mhap.sh 339 > ./mhap.000339.out 2>&1
    ./mhap.sh 340 > ./mhap.000340.out 2>&1
    ./mhap.sh 341 > ./mhap.000341.out 2>&1
    ./mhap.sh 342 > ./mhap.000342.out 2>&1
    ./mhap.sh 343 > ./mhap.000343.out 2>&1
    ./mhap.sh 344 > ./mhap.000344.out 2>&1
    ./mhap.sh 345 > ./mhap.000345.out 2>&1
    ./mhap.sh 346 > ./mhap.000346.out 2>&1
    ./mhap.sh 347 > ./mhap.000347.out 2>&1
    ./mhap.sh 348 > ./mhap.000348.out 2>&1
    ./mhap.sh 349 > ./mhap.000349.out 2>&1
    ./mhap.sh 350 > ./mhap.000350.out 2>&1
    ./mhap.sh 351 > ./mhap.000351.out 2>&1
    ./mhap.sh 352 > ./mhap.000352.out 2>&1
    ./mhap.sh 353 > ./mhap.000353.out 2>&1
    ./mhap.sh 354 > ./mhap.000354.out 2>&1
    ./mhap.sh 355 > ./mhap.000355.out 2>&1
    ./mhap.sh 356 > ./mhap.000356.out 2>&1
    ./mhap.sh 357 > ./mhap.000357.out 2>&1
    ./mhap.sh 358 > ./mhap.000358.out 2>&1
    ./mhap.sh 359 > ./mhap.000359.out 2>&1
    ./mhap.sh 360 > ./mhap.000360.out 2>&1
    ./mhap.sh 361 > ./mhap.000361.out 2>&1
    ./mhap.sh 362 > ./mhap.000362.out 2>&1
    ./mhap.sh 363 > ./mhap.000363.out 2>&1
    ./mhap.sh 364 > ./mhap.000364.out 2>&1
    ./mhap.sh 365 > ./mhap.000365.out 2>&1
    ./mhap.sh 366 > ./mhap.000366.out 2>&1
    ./mhap.sh 367 > ./mhap.000367.out 2>&1
    ./mhap.sh 368 > ./mhap.000368.out 2>&1
    ./mhap.sh 369 > ./mhap.000369.out 2>&1
    ./mhap.sh 370 > ./mhap.000370.out 2>&1
    ./mhap.sh 371 > ./mhap.000371.out 2>&1
    ./mhap.sh 372 > ./mhap.000372.out 2>&1
    ./mhap.sh 373 > ./mhap.000373.out 2>&1
    ./mhap.sh 374 > ./mhap.000374.out 2>&1
    ./mhap.sh 375 > ./mhap.000375.out 2>&1
    ./mhap.sh 376 > ./mhap.000376.out 2>&1
    ./mhap.sh 377 > ./mhap.000377.out 2>&1
    ./mhap.sh 378 > ./mhap.000378.out 2>&1
    ./mhap.sh 379 > ./mhap.000379.out 2>&1
    ./mhap.sh 380 > ./mhap.000380.out 2>&1
    ./mhap.sh 381 > ./mhap.000381.out 2>&1
    ./mhap.sh 382 > ./mhap.000382.out 2>&1
    ./mhap.sh 383 > ./mhap.000383.out 2>&1
    ./mhap.sh 384 > ./mhap.000384.out 2>&1
    ./mhap.sh 385 > ./mhap.000385.out 2>&1
    ./mhap.sh 386 > ./mhap.000386.out 2>&1
    ./mhap.sh 387 > ./mhap.000387.out 2>&1
    ./mhap.sh 388 > ./mhap.000388.out 2>&1
    ./mhap.sh 389 > ./mhap.000389.out 2>&1
    ./mhap.sh 390 > ./mhap.000390.out 2>&1
    ./mhap.sh 391 > ./mhap.000391.out 2>&1
    ./mhap.sh 392 > ./mhap.000392.out 2>&1
    ./mhap.sh 393 > ./mhap.000393.out 2>&1
    ./mhap.sh 394 > ./mhap.000394.out 2>&1
    ./mhap.sh 395 > ./mhap.000395.out 2>&1
    ./mhap.sh 396 > ./mhap.000396.out 2>&1
    ./mhap.sh 397 > ./mhap.000397.out 2>&1
    ./mhap.sh 398 > ./mhap.000398.out 2>&1
    ./mhap.sh 399 > ./mhap.000399.out 2>&1
    ./mhap.sh 400 > ./mhap.000400.out 2>&1
    ./mhap.sh 401 > ./mhap.000401.out 2>&1
    ./mhap.sh 402 > ./mhap.000402.out 2>&1
    ./mhap.sh 403 > ./mhap.000403.out 2>&1
    ./mhap.sh 404 > ./mhap.000404.out 2>&1
    ./mhap.sh 405 > ./mhap.000405.out 2>&1
    ./mhap.sh 406 > ./mhap.000406.out 2>&1
    ./mhap.sh 407 > ./mhap.000407.out 2>&1
    ./mhap.sh 408 > ./mhap.000408.out 2>&1
    ./mhap.sh 409 > ./mhap.000409.out 2>&1
    ./mhap.sh 410 > ./mhap.000410.out 2>&1
    ./mhap.sh 411 > ./mhap.000411.out 2>&1
    ./mhap.sh 412 > ./mhap.000412.out 2>&1
    ./mhap.sh 413 > ./mhap.000413.out 2>&1
    ./mhap.sh 414 > ./mhap.000414.out 2>&1
    ./mhap.sh 415 > ./mhap.000415.out 2>&1
    ./mhap.sh 416 > ./mhap.000416.out 2>&1
    ./mhap.sh 417 > ./mhap.000417.out 2>&1
    ./mhap.sh 418 > ./mhap.000418.out 2>&1
    ./mhap.sh 419 > ./mhap.000419.out 2>&1
    ./mhap.sh 420 > ./mhap.000420.out 2>&1
    ./mhap.sh 421 > ./mhap.000421.out 2>&1
    ./mhap.sh 422 > ./mhap.000422.out 2>&1
    ./mhap.sh 423 > ./mhap.000423.out 2>&1
    ./mhap.sh 424 > ./mhap.000424.out 2>&1
    ./mhap.sh 425 > ./mhap.000425.out 2>&1
    ./mhap.sh 426 > ./mhap.000426.out 2>&1
    ./mhap.sh 427 > ./mhap.000427.out 2>&1
    ./mhap.sh 428 > ./mhap.000428.out 2>&1
    ./mhap.sh 429 > ./mhap.000429.out 2>&1
    ./mhap.sh 430 > ./mhap.000430.out 2>&1
    ./mhap.sh 431 > ./mhap.000431.out 2>&1
    ./mhap.sh 432 > ./mhap.000432.out 2>&1
    ./mhap.sh 433 > ./mhap.000433.out 2>&1
    ./mhap.sh 434 > ./mhap.000434.out 2>&1
    ./mhap.sh 435 > ./mhap.000435.out 2>&1
    ./mhap.sh 436 > ./mhap.000436.out 2>&1
    ./mhap.sh 437 > ./mhap.000437.out 2>&1
    ./mhap.sh 438 > ./mhap.000438.out 2>&1
    ./mhap.sh 439 > ./mhap.000439.out 2>&1
    ./mhap.sh 440 > ./mhap.000440.out 2>&1
    ./mhap.sh 441 > ./mhap.000441.out 2>&1
    ./mhap.sh 442 > ./mhap.000442.out 2>&1
    ./mhap.sh 443 > ./mhap.000443.out 2>&1
    ./mhap.sh 444 > ./mhap.000444.out 2>&1
    ./mhap.sh 445 > ./mhap.000445.out 2>&1
    ./mhap.sh 446 > ./mhap.000446.out 2>&1
    ./mhap.sh 447 > ./mhap.000447.out 2>&1
    ./mhap.sh 448 > ./mhap.000448.out 2>&1
    ./mhap.sh 449 > ./mhap.000449.out 2>&1
    ./mhap.sh 450 > ./mhap.000450.out 2>&1
    ./mhap.sh 451 > ./mhap.000451.out 2>&1
    ./mhap.sh 452 > ./mhap.000452.out 2>&1
    ./mhap.sh 453 > ./mhap.000453.out 2>&1
    ./mhap.sh 454 > ./mhap.000454.out 2>&1
    ./mhap.sh 455 > ./mhap.000455.out 2>&1
    ./mhap.sh 456 > ./mhap.000456.out 2>&1
    ./mhap.sh 457 > ./mhap.000457.out 2>&1
    ./mhap.sh 458 > ./mhap.000458.out 2>&1
    ./mhap.sh 459 > ./mhap.000459.out 2>&1
    ./mhap.sh 460 > ./mhap.000460.out 2>&1
    ./mhap.sh 461 > ./mhap.000461.out 2>&1
    ./mhap.sh 462 > ./mhap.000462.out 2>&1
    ./mhap.sh 463 > ./mhap.000463.out 2>&1
    ./mhap.sh 464 > ./mhap.000464.out 2>&1
    ./mhap.sh 465 > ./mhap.000465.out 2>&1
    ./mhap.sh 466 > ./mhap.000466.out 2>&1
    ./mhap.sh 467 > ./mhap.000467.out 2>&1
    ./mhap.sh 468 > ./mhap.000468.out 2>&1
    ./mhap.sh 469 > ./mhap.000469.out 2>&1
    ./mhap.sh 470 > ./mhap.000470.out 2>&1
    ./mhap.sh 471 > ./mhap.000471.out 2>&1
    ./mhap.sh 472 > ./mhap.000472.out 2>&1
    ./mhap.sh 473 > ./mhap.000473.out 2>&1
    ./mhap.sh 474 > ./mhap.000474.out 2>&1
    ./mhap.sh 475 > ./mhap.000475.out 2>&1
    ./mhap.sh 476 > ./mhap.000476.out 2>&1
    ./mhap.sh 477 > ./mhap.000477.out 2>&1
    ./mhap.sh 478 > ./mhap.000478.out 2>&1
    ./mhap.sh 479 > ./mhap.000479.out 2>&1
    ./mhap.sh 480 > ./mhap.000480.out 2>&1
    ./mhap.sh 481 > ./mhap.000481.out 2>&1
    ./mhap.sh 482 > ./mhap.000482.out 2>&1
    ./mhap.sh 483 > ./mhap.000483.out 2>&1
    ./mhap.sh 484 > ./mhap.000484.out 2>&1
    ./mhap.sh 485 > ./mhap.000485.out 2>&1
    ./mhap.sh 486 > ./mhap.000486.out 2>&1
    ./mhap.sh 487 > ./mhap.000487.out 2>&1
    ./mhap.sh 488 > ./mhap.000488.out 2>&1
    ./mhap.sh 489 > ./mhap.000489.out 2>&1
    ./mhap.sh 490 > ./mhap.000490.out 2>&1
    ./mhap.sh 491 > ./mhap.000491.out 2>&1
    ./mhap.sh 492 > ./mhap.000492.out 2>&1
    ./mhap.sh 493 > ./mhap.000493.out 2>&1
    ./mhap.sh 494 > ./mhap.000494.out 2>&1
    ./mhap.sh 495 > ./mhap.000495.out 2>&1
    ./mhap.sh 496 > ./mhap.000496.out 2>&1
    ./mhap.sh 497 > ./mhap.000497.out 2>&1
    ./mhap.sh 498 > ./mhap.000498.out 2>&1
    ./mhap.sh 499 > ./mhap.000499.out 2>&1
    ./mhap.sh 500 > ./mhap.000500.out 2>&1
    ./mhap.sh 501 > ./mhap.000501.out 2>&1
    ./mhap.sh 502 > ./mhap.000502.out 2>&1
    ./mhap.sh 503 > ./mhap.000503.out 2>&1
    ./mhap.sh 504 > ./mhap.000504.out 2>&1
    ./mhap.sh 505 > ./mhap.000505.out 2>&1
    ./mhap.sh 506 > ./mhap.000506.out 2>&1
    ./mhap.sh 507 > ./mhap.000507.out 2>&1
    ./mhap.sh 508 > ./mhap.000508.out 2>&1
    ./mhap.sh 509 > ./mhap.000509.out 2>&1
    ./mhap.sh 510 > ./mhap.000510.out 2>&1
    ./mhap.sh 511 > ./mhap.000511.out 2>&1

-- Finished on Wed Sep 29 12:29:07 2021 (1600 seconds) with 3339.508 GB free disk space
----------------------------------------
-- Found 511 mhap overlap output files.
-- Finished stage 'cor-mhapCheck', reset canuIteration.
----------------------------------------
-- Starting command on Wed Sep 29 12:29:08 2021 with 3339.507 GB free disk space

    cd correction
    /home/safs-data02/dennislab/sw/canu/2.1.1/bin/ovStoreConfig \
     -S ../P123.seqStore \
     -M 4-8 \
     -L ./1-overlapper/ovljob.files \
     -create ./P123.ovlStore.config \
     > ./P123.ovlStore.config.txt \
    2> ./P123.ovlStore.config.err

-- Finished on Wed Sep 29 12:29:12 2021 (4 seconds) with 3339.5 GB free disk space
----------------------------------------
--
-- Creating overlap store correction/P123.ovlStore using:
--      2 buckets
--      2 slices
--        using at most 1 GB memory each
-- Finished stage 'cor-overlapStoreConfigure', reset canuIteration.
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'ovB' concurrent execution on Wed Sep 29 12:29:12 2021 with 3339.5 GB free disk space (2 processes; 1 concurrently)

    cd correction/P123.ovlStore.BUILDING
    ./scripts/1-bucketize.sh 1 > ./logs/1-bucketize.000001.out 2>&1
    ./scripts/1-bucketize.sh 2 > ./logs/1-bucketize.000002.out 2>&1

-- Finished on Wed Sep 29 12:29:15 2021 (3 seconds) with 3339.497 GB free disk space
----------------------------------------
-- Overlap store bucketizer finished.
-- Finished stage 'cor-overlapStoreBucketizerCheck', reset canuIteration.
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'ovS' concurrent execution on Wed Sep 29 12:29:15 2021 with 3339.497 GB free disk space (2 processes; 1 concurrently)

    cd correction/P123.ovlStore.BUILDING
    ./scripts/2-sort.sh 1 > ./logs/2-sort.000001.out 2>&1
    ./scripts/2-sort.sh 2 > ./logs/2-sort.000002.out 2>&1

-- Finished on Wed Sep 29 12:29:18 2021 (3 seconds) with 3339.148 GB free disk space
----------------------------------------
-- Overlap store sorter finished.
-- Finished stage 'cor-overlapStoreSorterCheck', reset canuIteration.
----------------------------------------
-- Starting command on Wed Sep 29 12:29:18 2021 with 3339.148 GB free disk space

    cd correction
    /home/safs-data02/dennislab/sw/canu/2.1.1/bin/ovStoreIndexer \
      -O  ./P123.ovlStore.BUILDING \
      -S ../P123.seqStore \
      -C  ./P123.ovlStore.config \
      -delete \
    > ./P123.ovlStore.BUILDING.index.err 2>&1

-- Finished on Wed Sep 29 12:29:19 2021 (one second) with 3338.898 GB free disk space
----------------------------------------
-- Checking store.
----------------------------------------
-- Starting command on Wed Sep 29 12:29:19 2021 with 3338.895 GB free disk space

    cd correction
    /home/safs-data02/dennislab/sw/canu/2.1.1/bin/ovStoreDump \
     -S ../P123.seqStore \
     -O  ./P123.ovlStore \
     -counts \
     > ./P123.ovlStore/counts.dat 2> ./P123.ovlStore/counts.err

-- Finished on Wed Sep 29 12:29:20 2021 (one second) with 3338.616 GB free disk space
----------------------------------------
--
-- Overlap store 'correction/P123.ovlStore' successfully constructed.
-- Found 344466 overlaps for 32468 reads; 3280623 reads have no overlaps.
--
--
-- Purged 10.931 GB in 1227 overlap output files.
-- Finished stage 'cor-createOverlapStore', reset canuIteration.
-- Set corMinCoverage=4 based on read coverage of 200.
-- Global filter scores will be estimated.
-- Computing correction layouts.
----------------------------------------
-- Starting command on Wed Sep 29 12:29:34 2021 with 3343.546 GB free disk space

    cd correction
    /home/safs-data02/dennislab/sw/canu/2.1.1/bin/generateCorrectionLayouts \
      -S ../P123.seqStore \
      -O  ./P123.ovlStore \
      -C  ./P123.corStore.WORKING \
      -eC 80 \
    > ./P123.corStore.err 2>&1

-- Finished on Wed Sep 29 12:29:35 2021 (one second) with 3343.014 GB free disk space
----------------------------------------
-- Finished stage 'cor-buildCorrectionLayoutsConfigure', reset canuIteration.
-- Computing correction layouts.
----------------------------------------
-- Starting command on Wed Sep 29 12:29:35 2021 with 3343.014 GB free disk space

    cd correction/2-correction
    /home/safs-data02/dennislab/sw/canu/2.1.1/bin/filterCorrectionLayouts \
      -S  ../../P123.seqStore \
      -C     ../P123.corStore \
      -R      ./P123.readsToCorrect.WORKING \
      -cc 4 \
      -cl 1000 \
      -g  8000000 \
      -c  40 \
    > ./P123.readsToCorrect.err 2>&1

-- Finished on Wed Sep 29 12:29:37 2021 (2 seconds) with 3342.153 GB free disk space
----------------------------------------
--                             original      original
--                            raw reads     raw reads
--   category                w/overlaps  w/o/overlaps
--   -------------------- ------------- -------------
--   Number of Reads              32138       3280953
--   Number of Bases          156860478       1330746
--   Coverage                    19.608         0.166
--   Median                        2524             0
--   Mean                          4880             0
--   N50                           9207          5424
--   Minimum                       1000             0
--   Maximum                      90216         17766
--   
--                                        --------corrected---------  ----------rescued----------
--                             evidence                     expected                     expected
--   category                     reads            raw     corrected            raw     corrected
--   -------------------- -------------  ------------- -------------  ------------- -------------
--   Number of Reads               4487           2270          2270              0             0
--   Number of Bases           19374770        8550432        842152              0             0
--   Coverage                     2.422          1.069         0.105          0.000         0.000
--   Median                        2909           2570            58              0             0
--   Mean                          4317           3766           370              0             0
--   N50                           6587           5468          1601              0             0
--   Minimum                       1000           1000             1              0             0
--   Maximum                      90216          64821         17448              0             0
--   
--                        --------uncorrected--------
--                                           expected
--   category                       raw     corrected
--   -------------------- ------------- -------------
--   Number of Reads            3310821       3310821
--   Number of Bases          149640792         22682
--   Coverage                    18.705         0.003
--   Median                           0             0
--   Mean                            45             0
--   N50                           9455             0
--   Minimum                          0             0
--   Maximum                      90216         22682
--   
--   Maximum Memory          1334558936
-- Finished stage 'cor-filterCorrectionLayouts', reset canuIteration.
-- Using overlaps no worse than 0.5 fraction error for correcting reads (from corErrorRate parameter).
--
-- Configuring correction jobs:
--   Jobs limited to 8 GB per job (via option corMemory).
--   Reads estimated to need at most 1.242 GB for computation.
--   Leaving 6.758 GB memory for read data.
----------------------------------------
-- Starting command on Wed Sep 29 12:29:37 2021 with 3342.153 GB free disk space

    cd correction/2-correction
    ./correctReadsPartition.sh \
    > ./correctReadsPartition.err 2>&1

-- Finished on Wed Sep 29 12:29:38 2021 (one second) with 3342.153 GB free disk space
----------------------------------------
-- Using overlaps no worse than 0.5 fraction error for correcting reads (from corErrorRate parameter).
-- Finished stage 'cor-generateCorrectedReadsConfigure', reset canuIteration.
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'cor' concurrent execution on Wed Sep 29 12:29:38 2021 with 3342.153 GB free disk space (1 processes; 1 concurrently)

    cd correction/2-correction
    ./correctReads.sh 1 > ./correctReads.000001.out 2>&1

-- Finished on Wed Sep 29 12:30:02 2021 (24 seconds) with 3342.164 GB free disk space
----------------------------------------
-- Found 1 read correction output files.
-- Finished stage 'cor-generateCorrectedReadsCheck', reset canuIteration.
-- Found 1 read correction output files.
-- Finished stage 'cor-generateCorrectedReadsCheck', reset canuIteration.
--
-- Loading corrected reads into corStore and seqStore.
----------------------------------------
-- Starting command on Wed Sep 29 12:30:02 2021 with 3342.164 GB free disk space

    cd correction
    /home/safs-data02/dennislab/sw/canu/2.1.1/bin/loadCorrectedReads \
      -S ../P123.seqStore \
      -C ./P123.corStore \
      -L ./2-correction/corjob.files \
    >  ./P123.loadCorrectedReads.log \
    2> ./P123.loadCorrectedReads.err

-- Finished on Wed Sep 29 12:30:03 2021 (one second) with 3341.964 GB free disk space
----------------------------------------
----------------------------------------
-- Starting command on Wed Sep 29 12:30:03 2021 with 3341.964 GB free disk space

    cd .
    /home/safs-data02/dennislab/sw/canu/2.1.1/bin/sqStoreDumpMetaData \
      -S ./P123.seqStore \
      -corrected \
      -histogram \
      -lengths \
    > ./P123.seqStore/readlengths-obt.dat \
    2> ./P123.seqStore/readlengths-obt.err 

-- Finished on Wed Sep 29 12:30:03 2021 (like a bat out of hell) with 3341.964 GB free disk space
----------------------------------------
----------------------------------------
-- Starting command on Wed Sep 29 12:30:03 2021 with 3341.964 GB free disk space

    cd .
    /home/safs-data02/dennislab/sw/canu/2.1.1/bin/sqStoreDumpMetaData \
      -S ./P123.seqStore \
      -corrected \
      -histogram \
    > ./P123.seqStore/readlengths-obt.txt \
    2> ./P123.seqStore/readlengths-obt.err 

-- Finished on Wed Sep 29 12:30:03 2021 (furiously fast) with 3341.964 GB free disk space
----------------------------------------
--
-- In sequence store './P123.seqStore':
--   Found 230 reads.
--   Found 397079 bases (0.04 times coverage).
--    
--    G=397079                           sum of  ||               length     num
--    NG         length     index       lengths  ||                range    seqs
--    ----- ------------ --------- ------------  ||  ------------------- -------
--    00010         9386         3        48914  ||         61-380            37|--------------------------------------------------------
--    00020         6420         8        85484  ||        381-700            19|-----------------------------
--    00030         3809        16       120902  ||        701-1020           42|---------------------------------------------------------------
--    00040         3286        27       159474  ||       1021-1340           28|------------------------------------------
--    00050         2431        41       199222  ||       1341-1660           28|------------------------------------------
--    00060         2055        59       240153  ||       1661-1980           15|-----------------------
--    00070         1598        81       279172  ||       1981-2300           10|---------------
--    00080         1255       108       317966  ||       2301-2620           12|------------------
--    00090          917       146       358170  ||       2621-2940            6|---------
--    00100           61       229       397079  ||       2941-3260            5|--------
--    001.000x                 230       397079  ||       3261-3580            9|--------------
--                                               ||       3581-3900            4|------
--                                               ||       3901-4220            3|-----
--                                               ||       4221-4540            0|
--                                               ||       4541-4860            1|--
--                                               ||       4861-5180            0|
--                                               ||       5181-5500            1|--
--                                               ||       5501-5820            1|--
--                                               ||       5821-6140            0|
--                                               ||       6141-6460            1|--
--                                               ||       6461-6780            1|--
--                                               ||       6781-7100            1|--
--                                               ||       7101-7420            0|
--                                               ||       7421-7740            1|--
--                                               ||       7741-8060            0|
--                                               ||       8061-8380            0|
--                                               ||       8381-8700            0|
--                                               ||       8701-9020            1|--
--                                               ||       9021-9340            0|
--                                               ||       9341-9660            1|--
--                                               ||       9661-9980            0|
--                                               ||       9981-10300           0|
--                                               ||      10301-10620           0|
--                                               ||      10621-10940           0|
--                                               ||      10941-11260           1|--
--                                               ||      11261-11580           0|
--                                               ||      11581-11900           0|
--                                               ||      11901-12220           0|
--                                               ||      12221-12540           1|--
--                                               ||      12541-12860           0|
--                                               ||      12861-13180           0|
--                                               ||      13181-13500           0|
--                                               ||      13501-13820           0|
--                                               ||      13821-14140           0|
--                                               ||      14141-14460           0|
--                                               ||      14461-14780           0|
--                                               ||      14781-15100           0|
--                                               ||      15101-15420           0|
--                                               ||      15421-15740           0|
--                                               ||      15741-16060           1|--
--    
--
-- Purging correctReads output after loading into stores.
-- Purged 1 .cns outputs.
-- Purged 2 .out job log outputs.
--
-- Purging overlaps used for correction.
-- Finished stage 'cor-loadCorrectedReads', reset canuIteration.
----------------------------------------
-- Starting command on Wed Sep 29 12:30:03 2021 with 3342.248 GB free disk space

    cd .
    /home/safs-data02/dennislab/sw/canu/2.1.1/bin/sqStoreDumpFASTQ \
      -corrected \
      -S ./P123.seqStore \
      -o ./P123.correctedReads.gz \
      -fasta \
      -nolibname \
    > P123.correctedReads.fasta.err 2>&1

-- Finished on Wed Sep 29 12:30:03 2021 (lickety-split) with 3342.248 GB free disk space
----------------------------------------
--
-- Corrected reads saved in 'P123.correctedReads.fasta.gz'.
-- Finished stage 'cor-dumpCorrectedReads', reset canuIteration.
--
--
-- BEGIN TRIMMING
--
--
-- ERROR:  Read coverage (0.04) lower than allowed.
-- ERROR:    minInputCoverage  = 10
-- ERROR:    stopOnLowCoverage = 10
-- ERROR:
-- ERROR:  This could be caused by an incorrect genomeSize or poor quality
-- ERROR:  reads that cound not be sufficiently corrected.
-- ERROR:
-- ERROR:  You can force Canu to continue by decreasing parameters
-- ERROR:  minInputCoverage and/or stopOnLowCoverage.  Be warned that the
-- ERROR:  quality of corrected reads and/or contiguity of contigs will be poor.
-- 

ABORT:
ABORT: canu 2.1.1
ABORT: Don't panic, but a mostly harmless error occurred and Canu stopped.
ABORT: Try restarting.  If that doesn't work, ask for help.
ABORT:
skoren commented 2 years ago

You ended up with no corrected reads. Is this a metagenomic dataset or clonal? If it's metagenomic, the issue is probably the default downsampling to 200x coverage Canu has. You'd want to then follow the FAQ for meta genomes: maxInputCoverage=10000 corOutCoverage=10000 corMinCoverage=0 redMemory=32 oeaMemory=32 batMemory=200. You'd have to remove the current assembly and start from scratch.

kattweigh commented 2 years ago

Yes it is a metagenomic dataset. I tried again using the script below:

load_canu
canu gridOptions="-p safs" maxInputCoverage=10000 corOutCoverage=10000 corMinCoverage=0 redMemory=32 oeaMemory=32 batMemory=200 minThreads=16 -d /safs-data01/uqkweigh/long-read-assemblies/canu_meta_P123/ -p P123 -genomeSize=8m -nanopore /safs-data02/dennislab/Public/Hoosfield/metagenomes/promethion/PromethION_FlipFlop_Garvin_27Mar20/coassemblies/allnanopore_P123.fq

And received an error that mhap failed. I then saw that there are two jobs waiting for resources in the queue. One entitled 'canuP123' with 1 core and 4G and the other cormhap... (the end of the job name is then cut off) with 16 cores and 384M. Could this be why the mhap failed? Is there a better way to run it to ensure these steps can run when they are required?

The following error was found in precompute.11037668_9.out:

Found perl:
   /usr/bin/perl
   This is perl 5, version 16, subversion 3 (v5.16.3) built for x86_64-linux-thread-multi

Found java:
   /usr/bin/java
   openjdk version "1.8.0_292"

Found canu:
   /home/safs-data02/dennislab/sw/canu/2.1.1/bin/canu
   canu 2.1.1

Running job 10 based on SLURM_ARRAY_TASK_ID=10 and offset=0.
/home/safs-data01/uqkweigh/long-read-assemblies/canu_meta_P123/correction/1-overlapper
Opened seqStore '../../P123.seqStore' for 'raw' reads.
Dumping raw reads from 145801 to 162000 (inclusive).

Starting mhap precompute.

Error occurred during initialization of VM
Unable to allocate 176960KB bitmaps for parallel garbage collection for the requested 5662720KB heap.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Mhap failed.
skoren commented 2 years ago

Isn't that exactly the same error as before, your JVM is mis-configured and doesn't work on grid job properly. Canu is correctly requesting 384*16 = 6gb but the JVM is still failing to allocate a heap less than 6gb (5.6gb to be exact). You can still use useGrid=false like you did to make it run before. Alternatively, you could also try increasing the memory allocation via gridOptionscormhap="--mem-per-cpu=1g" assuming the JVM is just over-allocating memory. Either way, kill all the jobs you have in the grid from canu.

kattweigh commented 2 years ago

Ah yes! Sorry for the oversight. I now have it working well. Thanks very much for all of your help :)