search

bioinfologics / w2rap-contigger

An Illumina PE genome contig assembler, can handle large (17Gbp) complex (hexaploid) genomes.
http://bioinfologics.github.io/the-w2rap-contigger/
MIT License
44 stars 14 forks source link

Step 1 Speed for w2rap contigger #33

Closed sanjitsbatra closed 6 years ago

sanjitsbatra commented 6 years ago

Hey!

On running a profiler on the w2rap-contigger, one sees that ~98.5% of the time is being spent in the function: PQVecEncoder::init. It is currently taking really long times to load reads into memory. Is there any way you can recommend speeding this up?

Thanks

bjclavijo commented 6 years ago

That is actually to be expected, as that step is doing an RLE encoding of the reads while everything else on step 1 is just bitencoding. This step 1 shouldn't be the most time consuming part of the assembly anyway so I wouldn't particularly worry about that. Can you try running with a smaller dataset and checking if that step is the critical one and is indeed unreasonably slow in your setup?

sanjitsbatra commented 6 years ago

On checking the Top progress of the w2rap-contigger, we see that the CPU usage is not crossing 100% which means that it is using only 1 core. I think this is the root cause of this tremendous slowdown. Could you please suggest why this might be happening and how we might go about fixing this?

Might these flags: export OMP_PROC_BIND=spread export MALLOC_PER_THREAD=1

have something to do with this?

bjclavijo commented 6 years ago

This was solved by the user by changing OMP parameters to match his own architecture better.