search

isovic / racon

Ultrafast consensus module for raw de novo genome assembly of long uncorrected reads. http://genome.cshlp.org/content/early/2017/01/18/gr.214270.116 Note: This was the original repository which will no longer be officially maintained. Please use the new official repository here:
https://github.com/lbcb-sci/racon
MIT License
271 stars 49 forks source link

Racon hanging on aligned overlap on unix #110

Open EpiBlake opened 5 years ago

EpiBlake commented 5 years ago

Hello,

There are some other issue reports that are similar such as racon stuck at generating consensus and I'm not sure if Racon is stalled or just taking a long time, but I wanted to reach out as the issue I am seeing is a bit different and is stalling at a different step.

I am running racon version 1.3.2 on a 28-core, 256Gb RAM unix server running CentOS 7. When I run racon, the program loads the sequences and the overlaps just fine, but seems to stall out/run very slow on the aligned overlap step. I have also seen this issue using v1.3.1.

Screen Shot 2019-03-25 at 3 47 04 PM

When I check the CPU usage of racon using top, it seems the program has gone to sleep or into some sort of dormancy.

Screen Shot 2019-03-25 at 3 47 24 PM

If I watch top, it will sometimes spike up to 0.2% CPU usage, but stays at 0.0% usage most of the time. Racon will continue to align overlaps, but it moves incredibly slowly, sometimes taking multiple hours.

If I run the exact same data downloaded from the unix server on a 4-core MacBook Pro with 16Gb of RAM, also using version 1.3.2 of racon, the program will complete in under an hour and will run full usage of the cores I give it. The same command is being used on both systems, just with a change in the threads parameter. I have also recreated this issue with multiple datasets now.

The command I am using is: racon -t 8 CombinedIlluminaReads.fq align.sam assembly.fasta > assembly_racon.fasta

I have tried building racon on the unix server using the git clone method outlined in the README, as well as using the release package tar.gz. Additionally, I have built the tool using the -Dracon_build_tests=ON flag and racon_tests passes all of the tests.

Any thoughts on what is going on?

rvaser commented 5 years ago

Hi, could it be that it is hitting swap due to memory shortage? How large are the datasets you recreated this issue? In some lower versions there was an overflow when using Illumina which caused an infinite loop, but that is fixed.

Best regards, Robert

EpiBlake commented 5 years ago

Hi Robert,

I would be shocked if this was hitting swap memory. The current example I posted above is running now and there is more than 28Gb of memory indicated as free. This also doesn't account for the memory bound up by the system that is freed when more memory is needed.

The dataset itself is not very big. The genome size is 2.7Mb and we have a little over 100x coverage for the Illumina reads. The assembly is from Oxford Nanopore data that has been nanopolished, and is now going through subsequent iterative racon polishes to get as clean of an assembly as possible.

One other note, I am using your script to combine the paired-end Illumina data.

Thanks for your help!

rvaser commented 5 years ago

Would you maybe be willing to share one of your datasets that behave as described above? It would make it easier for me to solve this issue.

EpiBlake commented 5 years ago

Absolutely. Which files would you like and is there a good way for me to transfer this to you directly?

rvaser commented 5 years ago

It would be the best if I get all files you were using (reads, overlaps, contigs) and the exact command. Share with google drive/dropbox and send the link via mail?

EpiBlake commented 5 years ago

Will do. Pulling it all together now. Thank you again!