useGrid=remote runs locally on Canu 1.7

[zyndagj]

Long running tasks are terminated on our login nodes, so I am trying to generate the sbatch scripts with useGrid=remote. Even though Canu correctly detects my slurm configuration, it just starts running tasks locally instead of generating slurm scripts.

Please let me know if I am running it incorrectly.

Thanks, Greg zynda

Command used for invocation:

$ canu -p ecoli -d ecoli-pacbio2 genomeSize=4.8m useGrid=remote -pacbio-raw pacbio.fastq

Output:

-- Canu 1.7
--
-- CITATIONS
--
-- Koren S, Walenz BP, Berlin K, Miller JR, Phillippy AM.
-- Canu: scalable and accurate long-read assembly via adaptive k-mer weighting and repeat separation.
-- Genome Res. 2017 May;27(5):722-736.
-- http://doi.org/10.1101/gr.215087.116
-- 
-- Read and contig alignments during correction, consensus and GFA building use:
--   Šošic M, Šikic M.
--   Edlib: a C/C ++ library for fast, exact sequence alignment using edit distance.
--   Bioinformatics. 2017 May 1;33(9):1394-1395.
--   http://doi.org/10.1093/bioinformatics/btw753
-- 
-- Overlaps are generated using:
--   Berlin K, et al.
--   Assembling large genomes with single-molecule sequencing and locality-sensitive hashing.
--   Nat Biotechnol. 2015 Jun;33(6):623-30.
--   http://doi.org/10.1038/nbt.3238
-- 
--   Myers EW, et al.
--   A Whole-Genome Assembly of Drosophila.
--   Science. 2000 Mar 24;287(5461):2196-204.
--   http://doi.org/10.1126/science.287.5461.2196
-- 
--   Li H.
--   Minimap and miniasm: fast mapping and de novo assembly for noisy long sequences.
--   Bioinformatics. 2016 Jul 15;32(14):2103-10.
--   http://doi.org/10.1093/bioinformatics/btw152
-- 
-- Corrected read consensus sequences are generated using an algorithm derived from FALCON-sense:
--   Chin CS, et al.
--   Phased diploid genome assembly with single-molecule real-time sequencing.
--   Nat Methods. 2016 Dec;13(12):1050-1054.
--   http://doi.org/10.1038/nmeth.4035
-- 
-- Contig consensus sequences are generated using an algorithm derived from pbdagcon:
--   Chin CS, et al.
--   Nonhybrid, finished microbial genome assemblies from long-read SMRT sequencing data.
--   Nat Methods. 2013 Jun;10(6):563-9
--   http://doi.org/10.1038/nmeth.2474
-- 
-- CONFIGURE CANU
--
-- Detected Java(TM) Runtime Environment '1.8.0_161' (from 'java').
-- Detected gnuplot version '4.6 patchlevel 2' (from 'gnuplot') and image format 'png'.
-- Detected 96 CPUs and 187 gigabytes of memory.
-- Detected Slurm with 'sinfo' binary in /bin/sinfo.
-- Detected Slurm with 'MaxArraySize' limited to 999 jobs.
-- 
-- Found 1680 hosts with  96 cores and  187 GB memory under Slurm control.
--
--                     (tag)Threads
--            (tag)Memory         |
--        (tag)         |         |  algorithm
--        -------  ------  --------  -----------------------------
-- Grid:  meryl      8 GB    4 CPUs  (k-mer counting)
-- Grid:  cormhap    6 GB   16 CPUs  (overlap detection with mhap)
-- Grid:  obtovl     4 GB    8 CPUs  (overlap detection)
-- Grid:  utgovl     4 GB    8 CPUs  (overlap detection)
-- Grid:  ovb        2 GB    1 CPU   (overlap store bucketizer)
-- Grid:  ovs        8 GB    1 CPU   (overlap store sorting)
-- Grid:  red        4 GB    4 CPUs  (read error detection)
-- Grid:  oea        4 GB    1 CPU   (overlap error adjustment)
-- Grid:  bat       16 GB    4 CPUs  (contig construction)
-- Grid:  gfa        8 GB    4 CPUs  (GFA alignment and processing)
--
-- Found PacBio uncorrected reads in the input files.
--
-- Generating assembly 'ecoli' in '/scratch/03076/gzynda/canu/ecoli-pacbio2'
--
-- Parameters:
--
--  genomeSize        4800000
--
--  Overlap Generation Limits:
--    corOvlErrorRate 0.2400 ( 24.00%)
--    obtOvlErrorRate 0.0450 (  4.50%)
--    utgOvlErrorRate 0.0450 (  4.50%)
--
--  Overlap Processing Limits:
--    corErrorRate    0.3000 ( 30.00%)
--    obtErrorRate    0.0450 (  4.50%)
--    utgErrorRate    0.0450 (  4.50%)
--    cnsErrorRate    0.0750 (  7.50%)
--
--
-- BEGIN CORRECTION
--
----------------------------------------
-- Starting command on Fri Mar 16 12:32:30 2018 with 4286566.993 GB free disk space

    cd .
    /home1/03076/gzynda/apps/intel17/canu/1.7/bin/gatekeeperCreate \
      -minlength 1000 \
      -o ./ecoli.gkpStore.BUILDING \
      ./ecoli.gkpStore.gkp \
    > ./ecoli.gkpStore.BUILDING.err 2>&1

-- Finished on Fri Mar 16 12:32:31 2018 (2 seconds) with 4286565.229 GB free disk space
----------------------------------------
--
-- In gatekeeper store './ecoli.gkpStore':
--   Found 12528 reads.
--   Found 115899341 bases (24.14 times coverage).
--
--   Read length histogram (one '*' equals 20.62 reads):
--        0    999      0 
--     1000   1999   1444 **********************************************************************
--     2000   2999   1328 ****************************************************************
--     3000   3999   1065 ***************************************************
--     4000   4999    774 *************************************
--     5000   5999    668 ********************************
--     6000   6999    619 ******************************
--     7000   7999    618 *****************************
--     8000   8999    607 *****************************
--     9000   9999    560 ***************************
--    10000  10999    523 *************************
--    11000  11999    478 ***********************
--    12000  12999    429 ********************
--    13000  13999    379 ******************
--    14000  14999    366 *****************
--    15000  15999    353 *****************
--    16000  16999    329 ***************
--    17000  17999    297 **************
--    18000  18999    294 **************
--    19000  19999    283 *************
--    20000  20999    251 ************
--    21000  21999    195 *********
--    22000  22999    152 *******
--    23000  23999    132 ******
--    24000  24999     75 ***
--    25000  25999     66 ***
--    26000  26999     56 **
--    27000  27999     44 **
--    28000  28999     35 *
--    29000  29999     16 
--    30000  30999     21 *
--    31000  31999     18 
--    32000  32999     11 
--    33000  33999      8 
--    34000  34999      6 
--    35000  35999      6 
--    36000  36999     10 
--    37000  37999      2 
--    38000  38999      3 
--    39000  39999      2 
--    40000  40999      2 
--    41000  41999      2 
--    42000  42999      1 
--
-- Running jobs.  First attempt out of 2.
----------------------------------------
-- Starting 'meryl' concurrent execution on Fri Mar 16 12:32:31 2018 with 4286565.229 GB free disk space (1 processes; 1 concurrently)

    cd correction/0-mercounts
    ./meryl.sh 1 > ./meryl.000001.out 2>&1

[skoren]

It looks like you are running Canu within a slurm job already (which Canu checks by looking for a SLURM_JOB_ID). This is not what remote expects, it expects to run on the head node without a SLURM_JOB_ID and will print commands to submit.

Alternatively, run with default or useGrid=1 and it will submit the jobs for you rather than stopping and telling you what to submit.

[zyndagj]

You are correct, I sandboxed myself in an interactive job.

After unsetting SLURM_JOB_ID and SLURM_JOB_NAME for good measure, Canu did create the job scripts and then exit.

Thanks!