search

marbl / canu

A single molecule sequence assembler for genomes large and small.
http://canu.readthedocs.io/
649 stars 178 forks source link

CANU 1.8 hybrid between pacbio and nanopore reads #1204

Closed biowackysci closed 5 years ago

biowackysci commented 5 years ago

I have been trying to generate an hybrid assembly using Pacbio reads and Nanopore reads with CANU 1.8. The slurm commands that i use is this :

#!/bin/bash
#PBS -V
#PBS -N Assembly_SE3
#PBS -q batch
#PBS -l nodes=1:ppn=16
#PBS -l walltime=1000:00:00
#PBS -l mem=100g
#PBS -M saila.vishwanathan@ecodev.vic.gov.au 
#PBS -m abe

#cp /group/pasture/Saila/MinION/Impact04_run2/20180426_0452_impact04_SV_26april2018/fastq/pass/fastq/ $TMPDIR/
cd $TMPDIR/
cp /group/pasture/Saila/MinION/all_Minion_fastq/prometh_min_all.fastq $TMPDIR/
cp /group/pasture/Saila/Pacbio/all_Pacbio_fastq/all_pacbio.fastq $TMDIR/

#cd $PBS_O_WORKDIR

module load Java
module load canu
canu -p canu -d /group/pasture/Saila/Canu1.8 genomeSize=2.8g -fast maxThreads=16 useGrid=false stopOnLowCoverage=10 corMhapSensitivity=normal correctedErrorRate=0.105 corOutCoverage=1000 minOverlapLength=1000 -nanopore-raw /group/pasture/Saila/MinION/all_Minion_fastq/prometh_min_all.fastq -pacbio-raw /group/pasture/Saila/Pacbio/all_Pacbio_fastq/all_pacbio.fastq

cp -r $TMPDIR/Nano_pacbio /group/pasture/Saila/Canu1.8/

Output file 

cp: cannot create regular file ‘/all_pacbio.fastq’: Permission denied
-- Canu 1.8
--
-- CITATIONS
-- CONFIGURE CANU
--
-- Detected Java(TM) Runtime Environment '1.8.0_172' (from '/usr/local/EasyBuild/software/Java/1.8.0_172/bin/java') with -d64 support.
--
-- WARNING:
-- WARNING:  Failed to run gnuplot using command 'gnuplot'.
-- WARNING:  Plots will be disabled.
-- WARNING:
--
-- Detected 48 CPUs and 754 gigabytes of memory.
-- Limited to 16 CPUs from maxThreads option.
-- Detected Slurm with 'sinfo' binary in /usr/local/slurm/current/bin/sinfo.
-- Grid engine disabled per useGrid=false option.
--
--                            (tag)Concurrency
--                     (tag)Threads          |
--            (tag)Memory         |          |
--        (tag)         |         |          |     total usage     algorithm
--        -------  ------  --------   --------  -----------------  -----------------------------
-- Local: meryl     64 GB    8 CPUs x   2 jobs   128 GB   16 CPUs  (k-mer counting)
-- Local: hap       16 GB   16 CPUs x   1 job     16 GB   16 CPUs  (read-to-haplotype assignment)
-- Local: cormhap   48 GB   16 CPUs x   1 job     48 GB   16 CPUs  (overlap detection with mhap)
-- Local: obtmhap   48 GB   16 CPUs x   1 job     48 GB   16 CPUs  (overlap detection with mhap)
-- Local: utgmhap   48 GB   16 CPUs x   1 job     48 GB   16 CPUs  (overlap detection with mhap)
-- Local: ovb        4 GB    1 CPU  x  16 jobs    64 GB   16 CPUs  (overlap store bucketizer)
-- Local: ovs       32 GB    1 CPU  x  16 jobs   512 GB   16 CPUs  (overlap store sorting)
-- Local: red       16 GB    8 CPUs x   2 jobs    32 GB   16 CPUs  (read error detection)
-- Local: oea        8 GB    1 CPU  x  16 jobs   128 GB   16 CPUs  (overlap error adjustment)
-- Local: bat      512 GB   16 CPUs x   1 job    512 GB   16 CPUs  (contig construction with bogart)
-- Local: gfa       32 GB   16 CPUs x   1 job     32 GB   16 CPUs  (GFA alignment and processing)
--
-- In 'canu.seqStore', found Nanopore reads:
--   Raw:        7297349
--   Corrected:  0
--   Trimmed:    0
--
-- Generating assembly 'canu' in '/group/pasture/Saila/Canu1.8'
--
-- Parameters:
--
--  genomeSize        2800000000
--
--  Overlap Generation Limits:
--    corOvlErrorRate 0.3200 ( 32.00%)
--    obtOvlErrorRate 0.1200 ( 12.00%)
--    utgOvlErrorRate 0.1200 ( 12.00%)
--
--  Overlap Processing Limits:
--    corErrorRate    0.5000 ( 50.00%)
--    obtErrorRate    0.1200 ( 12.00%)
--    utgErrorRate    0.1200 ( 12.00%)
--    cnsErrorRate    0.2000 ( 20.00%)
--
--
-- BEGIN CORRECTION
--
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'cormhap' concurrent execution on Mon Jan  7 10:00:05 2019 with 38062.593 GB free disk space (135 processes; 1 concurrently)

    cd correction/1-overlapper
    ./mhap.sh 147 > ./mhap.000147.out 2>&1
    ./mhap.sh 148 > ./mhap.000148.out 2>&1
    ./mhap.sh 149 > ./mhap.000149.out 2>&1
    ./mhap.sh 150 > ./mhap.000150.out 2>&1
    ./mhap.sh 151 > ./mhap.000151.out 2>&1
    ./mhap.sh 152 > ./mhap.000152.out 2>&1
    ./mhap.sh 153 > ./mhap.000153.out 2>&1
    ./mhap.sh 154 > ./mhap.000154.out 2>&1
    ./mhap.sh 155 > ./mhap.000155.out 2>&1
    ./mhap.sh 156 > ./mhap.000156.out 2>&1
    ./mhap.sh 157 > ./mhap.000157.out 2>&1
    ./mhap.sh 158 > ./mhap.000158.out 2>&1
    ./mhap.sh 159 > ./mhap.000159.out 2>&1
    ./mhap.sh 160 > ./mhap.000160.out 2>&1
    ./mhap.sh 161 > ./mhap.000161.out 2>&1
    ./mhap.sh 162 > ./mhap.000162.out 2>&1
    ./mhap.sh 163 > ./mhap.000163.out 2>&1
    ./mhap.sh 164 > ./mhap.000164.out 2>&1
    ./mhap.sh 165 > ./mhap.000165.out 2>&1
    ./mhap.sh 166 > ./mhap.000166.out 2>&1
    ./mhap.sh 167 > ./mhap.000167.out 2>&1
    ./mhap.sh 168 > ./mhap.000168.out 2>&1
    ./mhap.sh 169 > ./mhap.000169.out 2>&1
    ./mhap.sh 170 > ./mhap.000170.out 2>&1
    ./mhap.sh 171 > ./mhap.000171.out 2>&1
    ./mhap.sh 172 > ./mhap.000172.out 2>&1
    ./mhap.sh 173 > ./mhap.000173.out 2>&1
    ./mhap.sh 174 > ./mhap.000174.out 2>&1
    ./mhap.sh 175 > ./mhap.000175.out 2>&1
    ./mhap.sh 176 > ./mhap.000176.out 2>&1
    ./mhap.sh 177 > ./mhap.000177.out 2>&1
    ./mhap.sh 178 > ./mhap.000178.out 2>&1
    ./mhap.sh 179 > ./mhap.000179.out 2>&1
    ./mhap.sh 180 > ./mhap.000180.out 2>&1
    ./mhap.sh 181 > ./mhap.000181.out 2>&1
    ./mhap.sh 182 > ./mhap.000182.out 2>&1
    ./mhap.sh 183 > ./mhap.000183.out 2>&1
    ./mhap.sh 184 > ./mhap.000184.out 2>&1
    ./mhap.sh 185 > ./mhap.000185.out 2>&1
    ./mhap.sh 186 > ./mhap.000186.out 2>&1
    ./mhap.sh 187 > ./mhap.000187.out 2>&1
    ./mhap.sh 188 > ./mhap.000188.out 2>&1
    ./mhap.sh 189 > ./mhap.000189.out 2>&1
    ./mhap.sh 190 > ./mhap.000190.out 2>&1
    ./mhap.sh 191 > ./mhap.000191.out 2>&1
    ./mhap.sh 192 > ./mhap.000192.out 2>&1
    ./mhap.sh 193 > ./mhap.000193.out 2>&1
    ./mhap.sh 194 > ./mhap.000194.out 2>&1
    ./mhap.sh 195 > ./mhap.000195.out 2>&1
    ./mhap.sh 196 > ./mhap.000196.out 2>&1
    ./mhap.sh 197 > ./mhap.000197.out 2>&1
    ./mhap.sh 198 > ./mhap.000198.out 2>&1
    ./mhap.sh 199 > ./mhap.000199.out 2>&1
    ./mhap.sh 200 > ./mhap.000200.out 2>&1
    ./mhap.sh 201 > ./mhap.000201.out 2>&1
    ./mhap.sh 202 > ./mhap.000202.out 2>&1
    ./mhap.sh 203 > ./mhap.000203.out 2>&1
    ./mhap.sh 204 > ./mhap.000204.out 2>&1
    ./mhap.sh 205 > ./mhap.000205.out 2>&1
    ./mhap.sh 206 > ./mhap.000206.out 2>&1
    ./mhap.sh 207 > ./mhap.000207.out 2>&1
    ./mhap.sh 208 > ./mhap.000208.out 2>&1
    ./mhap.sh 209 > ./mhap.000209.out 2>&1
    ./mhap.sh 210 > ./mhap.000210.out 2>&1
    ./mhap.sh 211 > ./mhap.000211.out 2>&1
    ./mhap.sh 212 > ./mhap.000212.out 2>&1
    ./mhap.sh 213 > ./mhap.000213.out 2>&1
    ./mhap.sh 214 > ./mhap.000214.out 2>&1
    ./mhap.sh 215 > ./mhap.000215.out 2>&1
    ./mhap.sh 216 > ./mhap.000216.out 2>&1
    ./mhap.sh 217 > ./mhap.000217.out 2>&1
    ./mhap.sh 218 > ./mhap.000218.out 2>&1
    ./mhap.sh 219 > ./mhap.000219.out 2>&1
    ./mhap.sh 220 > ./mhap.000220.out 2>&1
    ./mhap.sh 221 > ./mhap.000221.out 2>&1
    ./mhap.sh 222 > ./mhap.000222.out 2>&1
    ./mhap.sh 223 > ./mhap.000223.out 2>&1
    ./mhap.sh 224 > ./mhap.000224.out 2>&1
    ./mhap.sh 225 > ./mhap.000225.out 2>&1
    ./mhap.sh 226 > ./mhap.000226.out 2>&1
    ./mhap.sh 227 > ./mhap.000227.out 2>&1
    ./mhap.sh 228 > ./mhap.000228.out 2>&1
    ./mhap.sh 229 > ./mhap.000229.out 2>&1
    ./mhap.sh 230 > ./mhap.000230.out 2>&1
    ./mhap.sh 231 > ./mhap.000231.out 2>&1
    ./mhap.sh 232 > ./mhap.000232.out 2>&1
    ./mhap.sh 233 > ./mhap.000233.out 2>&1
    ./mhap.sh 234 > ./mhap.000234.out 2>&1
    ./mhap.sh 235 > ./mhap.000235.out 2>&1
    ./mhap.sh 236 > ./mhap.000236.out 2>&1
    ./mhap.sh 237 > ./mhap.000237.out 2>&1
    ./mhap.sh 238 > ./mhap.000238.out 2>&1
    ./mhap.sh 239 > ./mhap.000239.out 2>&1
    ./mhap.sh 240 > ./mhap.000240.out 2>&1
    ./mhap.sh 241 > ./mhap.000241.out 2>&1
    ./mhap.sh 242 > ./mhap.000242.out 2>&1
    ./mhap.sh 243 > ./mhap.000243.out 2>&1
    ./mhap.sh 244 > ./mhap.000244.out 2>&1
    ./mhap.sh 245 > ./mhap.000245.out 2>&1
    ./mhap.sh 246 > ./mhap.000246.out 2>&1
    ./mhap.sh 247 > ./mhap.000247.out 2>&1
    ./mhap.sh 248 > ./mhap.000248.out 2>&1
    ./mhap.sh 249 > ./mhap.000249.out 2>&1
    ./mhap.sh 250 > ./mhap.000250.out 2>&1
    ./mhap.sh 251 > ./mhap.000251.out 2>&1
    ./mhap.sh 252 > ./mhap.000252.out 2>&1
    ./mhap.sh 253 > ./mhap.000253.out 2>&1
    ./mhap.sh 254 > ./mhap.000254.out 2>&1
    ./mhap.sh 255 > ./mhap.000255.out 2>&1
    ./mhap.sh 256 > ./mhap.000256.out 2>&1
    ./mhap.sh 257 > ./mhap.000257.out 2>&1
    ./mhap.sh 258 > ./mhap.000258.out 2>&1
    ./mhap.sh 259 > ./mhap.000259.out 2>&1
    ./mhap.sh 260 > ./mhap.000260.out 2>&1
    ./mhap.sh 261 > ./mhap.000261.out 2>&1
    ./mhap.sh 262 > ./mhap.000262.out 2>&1
    ./mhap.sh 263 > ./mhap.000263.out 2>&1
    ./mhap.sh 264 > ./mhap.000264.out 2>&1
    ./mhap.sh 265 > ./mhap.000265.out 2>&1
    ./mhap.sh 266 > ./mhap.000266.out 2>&1
    ./mhap.sh 267 > ./mhap.000267.out 2>&1
    ./mhap.sh 268 > ./mhap.000268.out 2>&1
    ./mhap.sh 269 > ./mhap.000269.out 2>&1
    ./mhap.sh 270 > ./mhap.000270.out 2>&1
    ./mhap.sh 271 > ./mhap.000271.out 2>&1
    ./mhap.sh 272 > ./mhap.000272.out 2>&1
    ./mhap.sh 273 > ./mhap.000273.out 2>&1
    ./mhap.sh 274 > ./mhap.000274.out 2>&1
    ./mhap.sh 275 > ./mhap.000275.out 2>&1
    ./mhap.sh 276 > ./mhap.000276.out 2>&1
    ./mhap.sh 277 > ./mhap.000277.out 2>&1
    ./mhap.sh 278 > ./mhap.000278.out 2>&1
    ./mhap.sh 279 > ./mhap.000279.out 2>&1
    ./mhap.sh 280 > ./mhap.000280.out 2>&1
    ./mhap.sh 281 > ./mhap.000281.out 2>&1

-- Finished on Mon Jan  7 10:00:19 2019 (14 seconds) with 38062.589 GB free disk space
----------------------------------------
--
-- Mhap overlap jobs failed, retry.
--   job correction/1-overlapper/results/000147.ovb FAILED.
--   job correction/1-overlapper/results/000148.ovb FAILED.
--   job correction/1-overlapper/results/000149.ovb FAILED.
--   job correction/1-overlapper/results/000150.ovb FAILED.
--   job correction/1-overlapper/results/000151.ovb FAILED.
--   job correction/1-overlapper/results/000152.ovb FAILED.
--   job correction/1-overlapper/results/000153.ovb FAILED.
--   job correction/1-overlapper/results/000154.ovb FAILED.
--   job correction/1-overlapper/results/000155.ovb FAILED.
--   job correction/1-overlapper/results/000156.ovb FAILED.
--   job correction/1-overlapper/results/000157.ovb FAILED.
--   job correction/1-overlapper/results/000158.ovb FAILED.
--   job correction/1-overlapper/results/000159.ovb FAILED.
--   job correction/1-overlapper/results/000160.ovb FAILED.
--   job correction/1-overlapper/results/000161.ovb FAILED.
--   job correction/1-overlapper/results/000162.ovb FAILED.
--   job correction/1-overlapper/results/000163.ovb FAILED.
--   job correction/1-overlapper/results/000164.ovb FAILED.
--   job correction/1-overlapper/results/000165.ovb FAILED.
--   job correction/1-overlapper/results/000166.ovb FAILED.
--   job correction/1-overlapper/results/000167.ovb FAILED.
--   job correction/1-overlapper/results/000168.ovb FAILED.
--   job correction/1-overlapper/results/000169.ovb FAILED.
--   job correction/1-overlapper/results/000170.ovb FAILED.
--   job correction/1-overlapper/results/000171.ovb FAILED.
--   job correction/1-overlapper/results/000172.ovb FAILED.
--   job correction/1-overlapper/results/000173.ovb FAILED.
--   job correction/1-overlapper/results/000174.ovb FAILED.
--   job correction/1-overlapper/results/000175.ovb FAILED.
--   job correction/1-overlapper/results/000176.ovb FAILED.
--   job correction/1-overlapper/results/000177.ovb FAILED.
--   job correction/1-overlapper/results/000178.ovb FAILED.
--   job correction/1-overlapper/results/000179.ovb FAILED.
--   job correction/1-overlapper/results/000180.ovb FAILED.
--   job correction/1-overlapper/results/000181.ovb FAILED.
--   job correction/1-overlapper/results/000182.ovb FAILED.
--   job correction/1-overlapper/results/000183.ovb FAILED.
--   job correction/1-overlapper/results/000184.ovb FAILED.
--   job correction/1-overlapper/results/000185.ovb FAILED.
--   job correction/1-overlapper/results/000186.ovb FAILED.
--   job correction/1-overlapper/results/000187.ovb FAILED.
--   job correction/1-overlapper/results/000188.ovb FAILED.
--   job correction/1-overlapper/results/000189.ovb FAILED.
--   job correction/1-overlapper/results/000190.ovb FAILED.
--   job correction/1-overlapper/results/000191.ovb FAILED.
--   job correction/1-overlapper/results/000192.ovb FAILED.
--   job correction/1-overlapper/results/000193.ovb FAILED.
--   job correction/1-overlapper/results/000194.ovb FAILED.
--   job correction/1-overlapper/results/000195.ovb FAILED.
--   job correction/1-overlapper/results/000196.ovb FAILED.
--   job correction/1-overlapper/results/000197.ovb FAILED.
--   job correction/1-overlapper/results/000198.ovb FAILED.
--   job correction/1-overlapper/results/000199.ovb FAILED.
--   job correction/1-overlapper/results/000200.ovb FAILED.
--   job correction/1-overlapper/results/000201.ovb FAILED.
--   job correction/1-overlapper/results/000202.ovb FAILED.
--   job correction/1-overlapper/results/000203.ovb FAILED.
--   job correction/1-overlapper/results/000204.ovb FAILED.
--   job correction/1-overlapper/results/000205.ovb FAILED.
--   job correction/1-overlapper/results/000206.ovb FAILED.
--   job correction/1-overlapper/results/000207.ovb FAILED.
--   job correction/1-overlapper/results/000208.ovb FAILED.
--   job correction/1-overlapper/results/000209.ovb FAILED.
--   job correction/1-overlapper/results/000210.ovb FAILED.
--   job correction/1-overlapper/results/000211.ovb FAILED.
--   job correction/1-overlapper/results/000212.ovb FAILED.
--   job correction/1-overlapper/results/000213.ovb FAILED.
--   job correction/1-overlapper/results/000214.ovb FAILED.
--   job correction/1-overlapper/results/000215.ovb FAILED.
--   job correction/1-overlapper/results/000216.ovb FAILED.
--   job correction/1-overlapper/results/000217.ovb FAILED.
--   job correction/1-overlapper/results/000218.ovb FAILED.
--   job correction/1-overlapper/results/000219.ovb FAILED.
--   job correction/1-overlapper/results/000220.ovb FAILED.
--   job correction/1-overlapper/results/000221.ovb FAILED.
--   job correction/1-overlapper/results/000222.ovb FAILED.
--   job correction/1-overlapper/results/000223.ovb FAILED.
--   job correction/1-overlapper/results/000224.ovb FAILED.
--   job correction/1-overlapper/results/000225.ovb FAILED.
--   job correction/1-overlapper/results/000226.ovb FAILED.
--   job correction/1-overlapper/results/000227.ovb FAILED.
--   job correction/1-overlapper/results/000228.ovb FAILED.
--   job correction/1-overlapper/results/000229.ovb FAILED.
--   job correction/1-overlapper/results/000230.ovb FAILED.
--   job correction/1-overlapper/results/000231.ovb FAILED.
--   job correction/1-overlapper/results/000232.ovb FAILED.
--   job correction/1-overlapper/results/000233.ovb FAILED.
--   job correction/1-overlapper/results/000234.ovb FAILED.
--   job correction/1-overlapper/results/000235.ovb FAILED.
--   job correction/1-overlapper/results/000236.ovb FAILED.
--   job correction/1-overlapper/results/000237.ovb FAILED.
--   job correction/1-overlapper/results/000238.ovb FAILED.
--   job correction/1-overlapper/results/000239.ovb FAILED.
--   job correction/1-overlapper/results/000240.ovb FAILED.
--   job correction/1-overlapper/results/000241.ovb FAILED.
--   job correction/1-overlapper/results/000242.ovb FAILED.
--   job correction/1-overlapper/results/000243.ovb FAILED.
--   job correction/1-overlapper/results/000244.ovb FAILED.
--   job correction/1-overlapper/results/000245.ovb FAILED.
--   job correction/1-overlapper/results/000246.ovb FAILED.
--   job correction/1-overlapper/results/000247.ovb FAILED.
--   job correction/1-overlapper/results/000248.ovb FAILED.
--   job correction/1-overlapper/results/000249.ovb FAILED.
--   job correction/1-overlapper/results/000250.ovb FAILED.
--   job correction/1-overlapper/results/000251.ovb FAILED.
--   job correction/1-overlapper/results/000252.ovb FAILED.
--   job correction/1-overlapper/results/000253.ovb FAILED.
--   job correction/1-overlapper/results/000254.ovb FAILED.
--   job correction/1-overlapper/results/000255.ovb FAILED.
--   job correction/1-overlapper/results/000256.ovb FAILED.
--   job correction/1-overlapper/results/000257.ovb FAILED.
--   job correction/1-overlapper/results/000258.ovb FAILED.
--   job correction/1-overlapper/results/000259.ovb FAILED.
--   job correction/1-overlapper/results/000260.ovb FAILED.
--   job correction/1-overlapper/results/000261.ovb FAILED.
--   job correction/1-overlapper/results/000262.ovb FAILED.
--   job correction/1-overlapper/results/000263.ovb FAILED.
--   job correction/1-overlapper/results/000264.ovb FAILED.
--   job correction/1-overlapper/results/000265.ovb FAILED.
--   job correction/1-overlapper/results/000266.ovb FAILED.
--   job correction/1-overlapper/results/000267.ovb FAILED.
--   job correction/1-overlapper/results/000268.ovb FAILED.
--   job correction/1-overlapper/results/000269.ovb FAILED.
--   job correction/1-overlapper/results/000270.ovb FAILED.
--   job correction/1-overlapper/results/000271.ovb FAILED.
--   job correction/1-overlapper/results/000272.ovb FAILED.
--   job correction/1-overlapper/results/000273.ovb FAILED.
--   job correction/1-overlapper/results/000274.ovb FAILED.
--   job correction/1-overlapper/results/000275.ovb FAILED.
--   job correction/1-overlapper/results/000276.ovb FAILED.
--   job correction/1-overlapper/results/000277.ovb FAILED.
--   job correction/1-overlapper/results/000278.ovb FAILED.
--   job correction/1-overlapper/results/000279.ovb FAILED.
--   job correction/1-overlapper/results/000280.ovb FAILED.
--   job correction/1-overlapper/results/000281.ovb FAILED.
--
--
-- Running jobs.  Second attempt out of 2.
----------------------------------------
-- Starting 'cormhap' concurrent execution on Mon Jan  7 10:00:19 2019 with 38062.589 GB free disk space (135 processes; 1 concurrently)

    cd correction/1-overlapper
    ./mhap.sh 147 > ./mhap.000147.out 2>&1
    ./mhap.sh 148 > ./mhap.000148.out 2>&1
    ./mhap.sh 149 > ./mhap.000149.out 2>&1
    ./mhap.sh 150 > ./mhap.000150.out 2>&1
    ./mhap.sh 151 > ./mhap.000151.out 2>&1
    ./mhap.sh 152 > ./mhap.000152.out 2>&1
    ./mhap.sh 153 > ./mhap.000153.out 2>&1
    ./mhap.sh 154 > ./mhap.000154.out 2>&1
    ./mhap.sh 155 > ./mhap.000155.out 2>&1
    ./mhap.sh 156 > ./mhap.000156.out 2>&1
    ./mhap.sh 157 > ./mhap.000157.out 2>&1
    ./mhap.sh 158 > ./mhap.000158.out 2>&1
    ./mhap.sh 159 > ./mhap.000159.out 2>&1
    ./mhap.sh 160 > ./mhap.000160.out 2>&1
    ./mhap.sh 161 > ./mhap.000161.out 2>&1
    ./mhap.sh 162 > ./mhap.000162.out 2>&1
    ./mhap.sh 163 > ./mhap.000163.out 2>&1
    ./mhap.sh 164 > ./mhap.000164.out 2>&1
    ./mhap.sh 165 > ./mhap.000165.out 2>&1
    ./mhap.sh 166 > ./mhap.000166.out 2>&1
    ./mhap.sh 167 > ./mhap.000167.out 2>&1
    ./mhap.sh 168 > ./mhap.000168.out 2>&1
    ./mhap.sh 169 > ./mhap.000169.out 2>&1
    ./mhap.sh 170 > ./mhap.000170.out 2>&1
    ./mhap.sh 171 > ./mhap.000171.out 2>&1
    ./mhap.sh 172 > ./mhap.000172.out 2>&1
    ./mhap.sh 173 > ./mhap.000173.out 2>&1
    ./mhap.sh 174 > ./mhap.000174.out 2>&1
    ./mhap.sh 175 > ./mhap.000175.out 2>&1
    ./mhap.sh 176 > ./mhap.000176.out 2>&1
    ./mhap.sh 177 > ./mhap.000177.out 2>&1
    ./mhap.sh 178 > ./mhap.000178.out 2>&1
    ./mhap.sh 179 > ./mhap.000179.out 2>&1
    ./mhap.sh 180 > ./mhap.000180.out 2>&1
    ./mhap.sh 181 > ./mhap.000181.out 2>&1
    ./mhap.sh 182 > ./mhap.000182.out 2>&1
    ./mhap.sh 183 > ./mhap.000183.out 2>&1
    ./mhap.sh 184 > ./mhap.000184.out 2>&1
    ./mhap.sh 185 > ./mhap.000185.out 2>&1
    ./mhap.sh 186 > ./mhap.000186.out 2>&1
    ./mhap.sh 187 > ./mhap.000187.out 2>&1
    ./mhap.sh 188 > ./mhap.000188.out 2>&1
    ./mhap.sh 189 > ./mhap.000189.out 2>&1
    ./mhap.sh 190 > ./mhap.000190.out 2>&1
    ./mhap.sh 191 > ./mhap.000191.out 2>&1
    ./mhap.sh 192 > ./mhap.000192.out 2>&1
    ./mhap.sh 193 > ./mhap.000193.out 2>&1
    ./mhap.sh 194 > ./mhap.000194.out 2>&1
    ./mhap.sh 195 > ./mhap.000195.out 2>&1
    ./mhap.sh 196 > ./mhap.000196.out 2>&1
    ./mhap.sh 197 > ./mhap.000197.out 2>&1
    ./mhap.sh 198 > ./mhap.000198.out 2>&1
    ./mhap.sh 199 > ./mhap.000199.out 2>&1
    ./mhap.sh 200 > ./mhap.000200.out 2>&1
    ./mhap.sh 201 > ./mhap.000201.out 2>&1
    ./mhap.sh 202 > ./mhap.000202.out 2>&1
    ./mhap.sh 203 > ./mhap.000203.out 2>&1
    ./mhap.sh 204 > ./mhap.000204.out 2>&1
    ./mhap.sh 205 > ./mhap.000205.out 2>&1
    ./mhap.sh 206 > ./mhap.000206.out 2>&1
    ./mhap.sh 207 > ./mhap.000207.out 2>&1
    ./mhap.sh 208 > ./mhap.000208.out 2>&1
    ./mhap.sh 209 > ./mhap.000209.out 2>&1
    ./mhap.sh 210 > ./mhap.000210.out 2>&1
    ./mhap.sh 211 > ./mhap.000211.out 2>&1
    ./mhap.sh 212 > ./mhap.000212.out 2>&1
    ./mhap.sh 213 > ./mhap.000213.out 2>&1
    ./mhap.sh 214 > ./mhap.000214.out 2>&1
    ./mhap.sh 215 > ./mhap.000215.out 2>&1
    ./mhap.sh 216 > ./mhap.000216.out 2>&1
    ./mhap.sh 217 > ./mhap.000217.out 2>&1
    ./mhap.sh 218 > ./mhap.000218.out 2>&1
    ./mhap.sh 219 > ./mhap.000219.out 2>&1
    ./mhap.sh 220 > ./mhap.000220.out 2>&1
    ./mhap.sh 221 > ./mhap.000221.out 2>&1
    ./mhap.sh 222 > ./mhap.000222.out 2>&1
    ./mhap.sh 223 > ./mhap.000223.out 2>&1
    ./mhap.sh 224 > ./mhap.000224.out 2>&1
    ./mhap.sh 225 > ./mhap.000225.out 2>&1
    ./mhap.sh 226 > ./mhap.000226.out 2>&1
    ./mhap.sh 227 > ./mhap.000227.out 2>&1
    ./mhap.sh 228 > ./mhap.000228.out 2>&1
    ./mhap.sh 229 > ./mhap.000229.out 2>&1
    ./mhap.sh 230 > ./mhap.000230.out 2>&1
    ./mhap.sh 231 > ./mhap.000231.out 2>&1
    ./mhap.sh 232 > ./mhap.000232.out 2>&1
    ./mhap.sh 233 > ./mhap.000233.out 2>&1
    ./mhap.sh 234 > ./mhap.000234.out 2>&1
    ./mhap.sh 235 > ./mhap.000235.out 2>&1
    ./mhap.sh 236 > ./mhap.000236.out 2>&1
    ./mhap.sh 237 > ./mhap.000237.out 2>&1
    ./mhap.sh 238 > ./mhap.000238.out 2>&1
    ./mhap.sh 239 > ./mhap.000239.out 2>&1
    ./mhap.sh 240 > ./mhap.000240.out 2>&1
    ./mhap.sh 241 > ./mhap.000241.out 2>&1
    ./mhap.sh 242 > ./mhap.000242.out 2>&1
    ./mhap.sh 243 > ./mhap.000243.out 2>&1
    ./mhap.sh 244 > ./mhap.000244.out 2>&1
    ./mhap.sh 245 > ./mhap.000245.out 2>&1
    ./mhap.sh 246 > ./mhap.000246.out 2>&1
    ./mhap.sh 247 > ./mhap.000247.out 2>&1
    ./mhap.sh 248 > ./mhap.000248.out 2>&1
    ./mhap.sh 249 > ./mhap.000249.out 2>&1
    ./mhap.sh 250 > ./mhap.000250.out 2>&1
    ./mhap.sh 251 > ./mhap.000251.out 2>&1
    ./mhap.sh 252 > ./mhap.000252.out 2>&1
    ./mhap.sh 253 > ./mhap.000253.out 2>&1
    ./mhap.sh 254 > ./mhap.000254.out 2>&1
    ./mhap.sh 255 > ./mhap.000255.out 2>&1
    ./mhap.sh 256 > ./mhap.000256.out 2>&1
    ./mhap.sh 257 > ./mhap.000257.out 2>&1
    ./mhap.sh 258 > ./mhap.000258.out 2>&1
    ./mhap.sh 259 > ./mhap.000259.out 2>&1
    ./mhap.sh 260 > ./mhap.000260.out 2>&1
    ./mhap.sh 261 > ./mhap.000261.out 2>&1
    ./mhap.sh 262 > ./mhap.000262.out 2>&1
    ./mhap.sh 263 > ./mhap.000263.out 2>&1
    ./mhap.sh 264 > ./mhap.000264.out 2>&1
    ./mhap.sh 265 > ./mhap.000265.out 2>&1
    ./mhap.sh 266 > ./mhap.000266.out 2>&1
    ./mhap.sh 267 > ./mhap.000267.out 2>&1
    ./mhap.sh 268 > ./mhap.000268.out 2>&1
    ./mhap.sh 269 > ./mhap.000269.out 2>&1
    ./mhap.sh 270 > ./mhap.000270.out 2>&1
    ./mhap.sh 271 > ./mhap.000271.out 2>&1
    ./mhap.sh 272 > ./mhap.000272.out 2>&1
    ./mhap.sh 273 > ./mhap.000273.out 2>&1
    ./mhap.sh 274 > ./mhap.000274.out 2>&1
    ./mhap.sh 275 > ./mhap.000275.out 2>&1
    ./mhap.sh 276 > ./mhap.000276.out 2>&1
    ./mhap.sh 277 > ./mhap.000277.out 2>&1
    ./mhap.sh 278 > ./mhap.000278.out 2>&1
    ./mhap.sh 279 > ./mhap.000279.out 2>&1
    ./mhap.sh 280 > ./mhap.000280.out 2>&1
    ./mhap.sh 281 > ./mhap.000281.out 2>&1

-- Finished on Mon Jan  7 10:00:33 2019 (14 seconds) with 38062.589 GB free disk space
----------------------------------------
--
-- Mhap overlap jobs failed, tried 2 times, giving up.
--   job correction/1-overlapper/results/000147.ovb FAILED.
--   job correction/1-overlapper/results/000148.ovb FAILED.
--   job correction/1-overlapper/results/000149.ovb FAILED.
--   job correction/1-overlapper/results/000150.ovb FAILED.
--   job correction/1-overlapper/results/000151.ovb FAILED.
--   job correction/1-overlapper/results/000152.ovb FAILED.
--   job correction/1-overlapper/results/000153.ovb FAILED.
--   job correction/1-overlapper/results/000154.ovb FAILED.
--   job correction/1-overlapper/results/000155.ovb FAILED.
--   job correction/1-overlapper/results/000156.ovb FAILED.
--   job correction/1-overlapper/results/000157.ovb FAILED.
--   job correction/1-overlapper/results/000158.ovb FAILED.
--   job correction/1-overlapper/results/000159.ovb FAILED.
--   job correction/1-overlapper/results/000160.ovb FAILED.
--   job correction/1-overlapper/results/000161.ovb FAILED.
--   job correction/1-overlapper/results/000162.ovb FAILED.
--   job correction/1-overlapper/results/000163.ovb FAILED.
--   job correction/1-overlapper/results/000164.ovb FAILED.
--   job correction/1-overlapper/results/000165.ovb FAILED.
--   job correction/1-overlapper/results/000166.ovb FAILED.
--   job correction/1-overlapper/results/000167.ovb FAILED.
--   job correction/1-overlapper/results/000168.ovb FAILED.
--   job correction/1-overlapper/results/000169.ovb FAILED.
--   job correction/1-overlapper/results/000170.ovb FAILED.
--   job correction/1-overlapper/results/000171.ovb FAILED.
--   job correction/1-overlapper/results/000172.ovb FAILED.
--   job correction/1-overlapper/results/000173.ovb FAILED.
--   job correction/1-overlapper/results/000174.ovb FAILED.
--   job correction/1-overlapper/results/000175.ovb FAILED.
--   job correction/1-overlapper/results/000176.ovb FAILED.
--   job correction/1-overlapper/results/000177.ovb FAILED.
--   job correction/1-overlapper/results/000178.ovb FAILED.
--   job correction/1-overlapper/results/000179.ovb FAILED.
--   job correction/1-overlapper/results/000180.ovb FAILED.
--   job correction/1-overlapper/results/000181.ovb FAILED.
--   job correction/1-overlapper/results/000182.ovb FAILED.
--   job correction/1-overlapper/results/000183.ovb FAILED.
--   job correction/1-overlapper/results/000184.ovb FAILED.
--   job correction/1-overlapper/results/000185.ovb FAILED.
--   job correction/1-overlapper/results/000186.ovb FAILED.
--   job correction/1-overlapper/results/000187.ovb FAILED.
--   job correction/1-overlapper/results/000188.ovb FAILED.
--   job correction/1-overlapper/results/000189.ovb FAILED.
--   job correction/1-overlapper/results/000190.ovb FAILED.
--   job correction/1-overlapper/results/000191.ovb FAILED.
--   job correction/1-overlapper/results/000192.ovb FAILED.
--   job correction/1-overlapper/results/000193.ovb FAILED.
--   job correction/1-overlapper/results/000194.ovb FAILED.
--   job correction/1-overlapper/results/000195.ovb FAILED.
--   job correction/1-overlapper/results/000196.ovb FAILED.
--   job correction/1-overlapper/results/000197.ovb FAILED.
--   job correction/1-overlapper/results/000198.ovb FAILED.
--   job correction/1-overlapper/results/000199.ovb FAILED.
--   job correction/1-overlapper/results/000200.ovb FAILED.
--   job correction/1-overlapper/results/000201.ovb FAILED.
--   job correction/1-overlapper/results/000202.ovb FAILED.
--   job correction/1-overlapper/results/000203.ovb FAILED.
--   job correction/1-overlapper/results/000204.ovb FAILED.
--   job correction/1-overlapper/results/000205.ovb FAILED.
--   job correction/1-overlapper/results/000206.ovb FAILED.
--   job correction/1-overlapper/results/000207.ovb FAILED.
--   job correction/1-overlapper/results/000208.ovb FAILED.
--   job correction/1-overlapper/results/000209.ovb FAILED.
--   job correction/1-overlapper/results/000210.ovb FAILED.
--   job correction/1-overlapper/results/000211.ovb FAILED.
--   job correction/1-overlapper/results/000212.ovb FAILED.
--   job correction/1-overlapper/results/000213.ovb FAILED.
--   job correction/1-overlapper/results/000214.ovb FAILED.
--   job correction/1-overlapper/results/000215.ovb FAILED.
--   job correction/1-overlapper/results/000216.ovb FAILED.
--   job correction/1-overlapper/results/000217.ovb FAILED.
--   job correction/1-overlapper/results/000218.ovb FAILED.
--   job correction/1-overlapper/results/000219.ovb FAILED.
--   job correction/1-overlapper/results/000220.ovb FAILED.
--   job correction/1-overlapper/results/000221.ovb FAILED.
--   job correction/1-overlapper/results/000222.ovb FAILED.
--   job correction/1-overlapper/results/000223.ovb FAILED.
--   job correction/1-overlapper/results/000224.ovb FAILED.
--   job correction/1-overlapper/results/000225.ovb FAILED.
--   job correction/1-overlapper/results/000226.ovb FAILED.
--   job correction/1-overlapper/results/000227.ovb FAILED.
--   job correction/1-overlapper/results/000228.ovb FAILED.
--   job correction/1-overlapper/results/000229.ovb FAILED.
--   job correction/1-overlapper/results/000230.ovb FAILED.
--   job correction/1-overlapper/results/000231.ovb FAILED.
--   job correction/1-overlapper/results/000232.ovb FAILED.
--   job correction/1-overlapper/results/000233.ovb FAILED.
--   job correction/1-overlapper/results/000234.ovb FAILED.
--   job correction/1-overlapper/results/000235.ovb FAILED.
--   job correction/1-overlapper/results/000236.ovb FAILED.
--   job correction/1-overlapper/results/000237.ovb FAILED.
--   job correction/1-overlapper/results/000238.ovb FAILED.
--   job correction/1-overlapper/results/000239.ovb FAILED.
--   job correction/1-overlapper/results/000240.ovb FAILED.
--   job correction/1-overlapper/results/000241.ovb FAILED.
--   job correction/1-overlapper/results/000242.ovb FAILED.
--   job correction/1-overlapper/results/000243.ovb FAILED.
--   job correction/1-overlapper/results/000244.ovb FAILED.
--   job correction/1-overlapper/results/000245.ovb FAILED.
--   job correction/1-overlapper/results/000246.ovb FAILED.
--   job correction/1-overlapper/results/000247.ovb FAILED.
--   job correction/1-overlapper/results/000248.ovb FAILED.
--   job correction/1-overlapper/results/000249.ovb FAILED.
--   job correction/1-overlapper/results/000250.ovb FAILED.
--   job correction/1-overlapper/results/000251.ovb FAILED.
--   job correction/1-overlapper/results/000252.ovb FAILED.
--   job correction/1-overlapper/results/000253.ovb FAILED.
--   job correction/1-overlapper/results/000254.ovb FAILED.
--   job correction/1-overlapper/results/000255.ovb FAILED.
--   job correction/1-overlapper/results/000256.ovb FAILED.
--   job correction/1-overlapper/results/000257.ovb FAILED.
--   job correction/1-overlapper/results/000258.ovb FAILED.
--   job correction/1-overlapper/results/000259.ovb FAILED.
--   job correction/1-overlapper/results/000260.ovb FAILED.
--   job correction/1-overlapper/results/000261.ovb FAILED.
--   job correction/1-overlapper/results/000262.ovb FAILED.
--   job correction/1-overlapper/results/000263.ovb FAILED.
--   job correction/1-overlapper/results/000264.ovb FAILED.
--   job correction/1-overlapper/results/000265.ovb FAILED.
--   job correction/1-overlapper/results/000266.ovb FAILED.
--   job correction/1-overlapper/results/000267.ovb FAILED.
--   job correction/1-overlapper/results/000268.ovb FAILED.
--   job correction/1-overlapper/results/000269.ovb FAILED.
--   job correction/1-overlapper/results/000270.ovb FAILED.
--   job correction/1-overlapper/results/000271.ovb FAILED.
--   job correction/1-overlapper/results/000272.ovb FAILED.
--   job correction/1-overlapper/results/000273.ovb FAILED.
--   job correction/1-overlapper/results/000274.ovb FAILED.
--   job correction/1-overlapper/results/000275.ovb FAILED.
--   job correction/1-overlapper/results/000276.ovb FAILED.
--   job correction/1-overlapper/results/000277.ovb FAILED.
--   job correction/1-overlapper/results/000278.ovb FAILED.
--   job correction/1-overlapper/results/000279.ovb FAILED.
--   job correction/1-overlapper/results/000280.ovb FAILED.
--   job correction/1-overlapper/results/000281.ovb FAILED.
--

ABORT:
ABORT: Canu 1.8
ABORT: Don't panic, but a mostly harmless error occurred and Canu stopped.
ABORT: Try restarting.  If that doesn't work, ask for help.
ABORT:
cp: cannot stat ‘/tmp/Nano_pacbio’: No such file or directory

I am running it on a grid

I am not sure of this error, also previously I was getting overlapping errors with this script. Thanks for your help in advance Saila

brianwalenz commented 5 years ago

What is in correction/1-overlapper/mhap.000147.out? Check the later numbered files for similar errors too.

(more comments in the other issue)

biowackysci commented 5 years ago

Also, what i dont understand here is that it does not read the Pacbio files in the beginning it reads In 'canu.seqStore', found Nanopore reads: -- Raw: 7297349 -- Corrected: 0 -- Trimmed: 0

But I dont see the Pacbio files.

Also i could not really open the correction/1-overlapper/mhap.000147.out files.

Is there something else I could do ?

Thanks

Saila

skoren commented 5 years ago

What is in the /group/pasture/Saila/Canu1.8/correction/1-overlapper folder?

biowackysci commented 5 years ago

the /group/pasture/Saila/Canu1.8/correction/1-overlapper folder has the following folders mhap.sh partioning.log precomputer.sh queries.tar precompute.000001.out precompute.000002.out core.264866 core.265561 mhap.000001.out mhap.000002.out

Thanks Saila

skoren commented 5 years ago

I don’t think that’s the right folder, your job was running up to partition 281 but you have only 2 here. Did you launch more than one run with the same Canu d parameter? What’s in mhap.sh and mhap.000001.out

brianwalenz commented 5 years ago

Idle.