andersen-lab / ivar

iVar is a computational package that contains functions broadly useful for viral amplicon-based sequencing.
https://andersen-lab.github.io/ivar/html/
GNU General Public License v3.0
115 stars 39 forks source link

Concurrency for `ivar trim`? #139

Open philiptzou opened 1 year ago

philiptzou commented 1 year ago

Hello, I'm wondering if you have plan to support multi-threads/workers in ivar?

cmaceves commented 1 year ago

Hi! Not the immediate future. Is there a specific case or dataset you have where you feel that would be vital? (ie. are you running into issues where process time is too long in the current implementation)

philiptzou commented 1 year ago

@cmaceves Thanks for the quick response. No I didn't run into any issues - but just discovered iVar is the slow bottleneck in our pipeline and I found the CPU usage of iVar never exceeds 100% (top -c).

I can imagine it is practical to use multiple CPUs to process a BAM file includes short reads, since you can just make chunks and deliver the chunks to sub-workers. That's the reason why I ask if you have plan to do so.

cmaceves commented 1 year ago

@philiptzou Yeah - I agree that definitely seems within reason and practical as a feature request for future implementation. We're getting ready for a release in the next month or so, but will keep this in mind for future milestones. Thanks for brining this to our attention!

philiptzou commented 1 year ago

@cmaceves Thank you!

khyox commented 1 year ago

(...) We're getting ready for a release in the next month or so, but will keep this in mind for future milestones. (...)

Hi @cmaceves. Are you still planning an imminent new release? Thanks!

cmaceves commented 1 year ago

@khyox hey! I'm caught up in doctorate applications at the moment, we're anticipating a release in December before the holiday season

khyox commented 1 year ago

Great news!! Thank you very much @cmaceves and best of luck with the applications! :)

khyox commented 1 year ago

Hey @cmaceves! Hope you were lucky with your PhD applications. I guess the release plans got delayed somehow. Are you still planning a new version soon? Thanks!

cmaceves commented 1 year ago

hi @khyox thank you, it went well. we're in the process of doing some final code reviews prior to release, it's still imminent!

khyox commented 1 year ago

Both great news! :) Thank you so much @cmaceves!