canu failed with 'failed to precompute mhap indices. Made 2 attempts, jobs still failed'.

[hjjansen]

When run locally Canu fails in the mhap precompute stage with 1 finished job and 4 to compute. The same data was used on the same machine in canu v1.0 to produce assemblies 1 to 7 without any problem. After installation of Canu v1.1 (+16 commits) r7394 a57a2edacb17f7dd842f32e2ef58f22953394b05 Canu stops with this message: "canu failed with 'failed to precompute mhap indices. Made 2 attempts, jobs still failed'."

In the correction/1-overlapper folder I can find the precompute.out files. /assembly8/correction/1-overlapper/precompute.000001.out is finished and looks like this:

Dumping reads from 1 to 9000 (inclusive).
Starting mhap precompute.
Running with these settings:
Version = 1.5b1
Build time = 01/22/2015 12:04 PM
--filter-threshold = 5.0E-6
--help = false
--max-seq-size = 100000
--max-shift = 0.2
--min-store-length = 499
--no-self = false
--num-hashes = 768
--num-min-matches = 2
--num-threads = 8
--pacbio_experimental = false
--pacbio_fast = false
--pacbio_sensitive = false
--store-full-id = false
--threshold = 0.04
--version = false
--weighted = true
-f = /mnt/data3/minion_data/zfg150502/assembly8/correction/0-mercounts/zfg150502_assembly8.ms16.frequentMers.ignore
-h = false
-k = 16
-p = /mnt/data3/minion_data/zfg150502/assembly8/correction/1-overlapper/blocks/000001.fasta
-q = /mnt/data3/minion_data/zfg150502/assembly8/correction/1-overlapper/blocks
-s =

Reading in filter file /mnt/data3/minion_data/zfg150502/assembly8/correction/0-mercounts/zfg150502_assembly8.ms16.frequentMers.ignore.
Time (s) to read filter file: 0.019249216
Processing FASTA files for binary compression...
Current # sequences loaded and processed from file: 5000...
Current # sequences loaded and processed from file: 10000...

/assembly8/correction/1-overlapper/precompute.000002.out fails and looks like this

ERROR:  AS_MAX_READS_BITS in store = 41, differs from executable = 37
ERROR:  AS_MAX_READLEN_BITS in store = 17, differs from executable = 21
ERROR:
ERROR:  Can't open store '/mnt/data3/minion_data/zfg150502/assembly8/correction/zfg150502_assembly8.gkpStore': parameters in src/AS_global.H are incompatible with the store.
mv: cannot stat ‘/mnt/data3/minion_data/zfg150502/assembly8/correction/1-overlapper/blocks/000002.fasta’: No such file or directory
Failed to extract fasta.

The zfg150502_assembly8.gkpStore was made successfully according to /assembly8/correction/zfg150502_assembly8.gkpStore.err which looked like this:

Starting file '/mnt/data3/minion_data/zfg150502/assembly8/correction/zfg150502_assembly8.gkpStore.gkp'.

  Loading reads from '/mnt/data3/minion_data/zfg150502/zfg150502_allruns_2d.fastq'
    Processed 207180 lines.
    Loaded 331948384 bp from:
      41436 FASTQ format reads (331948384 bp).
    WARNING: 532 reads (1.2839%) with 318121 bp (0.0957%) were too short (< 1000bp) and were ignored.

Finished with:
  0 warnings (bad base or qv, too short, too long)

Loaded into store:
  331948384 bp.
  40904 reads.

Skipped (too short):
  318121 bp (0.0957%).
  532 reads (1.2839%).

gatekeeperCreate finished successfully.

I'm not sure how to proceed to solve this error. Any guidance is appreciated.

Hans Jansen

[JohnUrban]

Did you update Canu in the middle of an assembly?

[hjjansen]

No, the machine wasn't used for anything other than this update.

[JohnUrban]

Lines like this made me curious: ERROR: AS_MAX_READS_BITS in store = 41, differs from executable = 37 ERROR: AS_MAX_READLEN_BITS in store = 17, differs from executable = 21

[hjjansen]

Me too but I don't have enough knowledge to see what I did wrong.

[skoren]

Your run is definitely mixing Canu v1.0 and v1.1, based both on the bits message and that pre-compute is using mhap 1.5 whereas Canu v1.1 should be using mhap 2.0. It seems like the run starts with Canu 1.0 and switches to 1.1 at some point when your error is reported. Do you have both installed on the system? Maybe one is being picked up in the path?

Can you remove your entire asm directory (/mnt/data3/minion_data/zfg150502/assembly8) and then post the full output of the Canu run up to the failure?

[hjjansen]

Thanks a million Sergey and apologies for wasting your time, this was entirely my mistake. I started the assembly8 with v1.0 and then decided to upgrade to v1.1 so I stopped Canu and upgraded but didn't delete the assembly8 directory thinking it would be overwritten by the new assembly8. I now deleted this directory and started the v1.1 assembler again. It now works like a charm. Thanks for this great tool!

Hans Jansen