zhanxw / rvtests

Rare variant test software for next generation sequencing data
131 stars 41 forks source link

thread count for rvtest #20

Closed CanWood closed 7 years ago

CanWood commented 7 years ago

Hi folks,

Although the rvtest binary seems to only use a single core for much of its work, we're seeing thread counts jump to take all available cores on the system. Is there a way to limit this? The context is so that we can know what resources to ask for when submitting rvtest jobs to HPC systems where the number of CPUs required must be provided.

Cheers

zhanxw commented 7 years ago

RVTESTS uses OpenMP. You can limit number of threads using: OMP_NUM_THREADS=XX before your rvtests commands.

Which version did you use? This behavior is not what I expected and I would like fix it.

On Thu, Apr 20, 2017 at 10:56 PM, CanWood notifications@github.com wrote:

Hi folks,

Although the rvtest binary seems to only use a single core for much of its work, we're seeing thread counts jump to take all available cores on the system. Is there a way to limit this? The context is so that we can know what resources to ask for when submitting rvtest jobs to HPC systems where the number of CPUs required must be provided.

Cheers

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/zhanxw/rvtests/issues/20, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJoiOcC8GbHG2L4KyHVlI8jmzxGgPbpks5ryCkIgaJpZM4ND1gS .

CanWood commented 7 years ago

Thanks for the quick response. We'll try the OMP_NUM_THREADS=1 and see how we go

Originally we were running rvtests (version: 20160630, git: Unknown) and it was running many cores for quite some time. We then tested with rvtests (version: 20170228, git: Unknown) and the length of time that it took all of the cores on the box was much decreased. when running the same commands

CanWood commented 7 years ago

OMP_NUM_THREADS did the trick. Now our users can have resource requests that are in line with the tool's use. Thanks!