TELR call runs exceptionally long with little progression

[renhamm]

I just ran TELR using the following command: telr -i /path/to/long/reads/1034.fastq -r /path/to/refgenome/Mguttatus_256_v2.0.fa -l /path/to/consensuslib/allTE-families.cons --out TELR_annotations

It ran for 10 full days, generating only the sam file and potentially non-fininshed ngm files in the "intermediate_files" directory. This seems exceptionally long for a single pacbio file of a 420Mb genome. Am I missing something?

The entire error output for the 10 day run is copied below: Writing output (SAM) to stdout Encoding reference sequence. Size of reference genome 314 Mbp (max. 68719 Mbp) 0 reference sequences were skipped (length < 10). Writing encoded reference to /global/scratch/users/laurenhamm/thesis/TELR_annotations/intermediate_files/Mguttatus_256_v2.0.fa-enc.2.ngm Writing to disk took 0.20s Building reference index #0 (kmer length: 13, reference skip: 2) 3100 prefixes were ignored due to the frequency cutoff (1000) Overall time for creating RefTable: 19.59s Writing reference index to /global/scratch/users/laurenhamm/thesis/TELR_annotations/intermediate_files/Mguttatus_256_v2.0.fa-ht-13-2.2.ngm Writing to disk took 0.62s Opening query file /global/scratch/users/laurenhamm/thesis/TELR_annotations/intermediate_files/1034.fastq No progress information (use --progress) Mapping reads... Debug: PositionNM reallocated. Debug: PositionNM reallocated. Debug: PositionNM reallocated. slurmstepd: error: JOB 12625093 ON n0006.savio2 CANCELLED AT 2022-08-18T15:03:03 DUE TO TIME LIMIT

[shunhuahan]

Hi @laurenhamm, thanks for reporting the issue!

• On first glance of the error output, it looks like the error was raised due to the TELR run taking longer than the time limit for the slurm job. I noticed that you didn't define the number of cores that can be used by TELR using -t option. Without this option, TELR would use only one thread for the entire run, which is likely causing this long running time issue.
• Also, just as a general reminder, you should also provide sufficient memory for the TELR run, given that NGMLR alignment is generally memory intensive (you could check the slurm job standard output for memory usage).
• I recommend re-running TELR with -t option and provide at least 10 cores for this run. If the run/NGMLR alignment is still still taking too long to finish, please consider running TELR with --aligner minimap2 option. Whole genome read alignment using minimap2 is significantly faster compared to using NGMLR (at the cost of some drops in alignment accuracy, especially in difficult regions).
• I also noticed the PositionNM reallocated debug messages generated by NGMLR, this issue has been reported at https://github.com/philres/ngmlr/issues/65 but the question hasn't been answered by the developer yet. I will keep an eye on it.
• Let me know if you have more questions, thanks!

Shunhua

[shunhuahan]

@laurenhamm Is the issue resolved by following the above suggestions? Let me know if you are still experiencing issues, or we can close the thread if it's resolved :)

[renhamm]

The threading plus minimap options fixed the timeout issue I was happening. I'm now hitting another issue, but I'll open a different issue for that. Thanks!