isovic / racon

Ultrafast consensus module for raw de novo genome assembly of long uncorrected reads. http://genome.cshlp.org/content/early/2017/01/18/gr.214270.116 Note: This was the original repository which will no longer be officially maintained. Please use the new official repository here:
https://github.com/lbcb-sci/racon
MIT License
261 stars 48 forks source link

Racon GPU floating point exeception #187

Open MRRedlinger opened 3 years ago

MRRedlinger commented 3 years ago

I'm encountering a floating point exception after the GPU memory allocation step on most, but not all, of my attempts to polish a reference with racon using the GPU. There are no issues when the -c flag isn't included.

I recently installed the GPU accelerated version of racon, per the instructions on this github page. I'm attempting to polish a 9kb reference genome using about 800k ONT reads.

I mapped the reads to the reference with minimap2, and passed the sam file along with the reads and reference to racon. For this polishing, the GPU accelerated racon worked perfectly.

For the second round of polishing I mapped the reads to the racon output from the previous round and passed the sam file, racon output and raw reads to racon. This time I encountered the floating point error.

This error is repeatable, with the reads above I can perform the 1st round of polishing repeatedly without error but the second round of polishing will always produce this error. For other barcodes I will get the error when attempting the first round of polishing.

We are using ubuntu 18.04 on with an AMD epyc CPU. There are two GPU's installed in the system, a Quadro RTX 6000 and an RTX 2080ti. racon appears to allocate memory on both when running with the -c flag.

rvaser commented 3 years ago

Hello, when you do not use the -c flag, are you using any other CUDA options like --cudaaligner-batches? If not, than it is GPU Racon issue. @tijyojwad, could please inspect this?

Best regards, Robert

MRRedlinger commented 3 years ago

No, when I'm not using the -c option I am only using the CPU.

Thank you, Matthew