search

Illumina / canvas

Canvas - Copy number variant (CNV) calling from DNA sequencing data
Other
121 stars 20 forks source link

Limiting number of threads to use #30

Open holtgrewe opened 7 years ago

holtgrewe commented 7 years ago

How can I limit the number of threads to use by Canvas? This is important in shared environments such as HPC clusters.

eroller commented 7 years ago

It's not possible yet, but it is something we are planning to do. I agree it is a useful feature for cluster computing.

dvav commented 6 years ago

Hi, I was wondering whether you have made any progress on this issue.

eroller commented 6 years ago

There have been improvements in the framework we are using to manage CPU/memory usage which should make this update easier, but we haven't had the time to update Canvas to utilize the new functionality. Specifically we need to add configuration options (or command line options) to specify maximum number of CPUs/memory and then wire those options up in each Canvas component. No clear timeline for this to get done, although I believe we will need to add this functionality in the beginning of next year.

Rashesh7 commented 6 years ago

Hi Eric,

Has this been implemented in the newest version? The Somatic-WGS workflow takes about 90GB of memory to run, which might be difficult to provide all the time.

Thanks, Rashesh

eroller commented 6 years ago

This still hasn't been implemented. The memory usage is dependent on the number of cores available on the machine. For high core machines with low amount of memory this could cause a problem.

DaGaMs commented 4 years ago

Hi guys,

The fact that Canvas just consumes all the cores on the machine makes it extremely difficult to use in a shared resource environment like our HPC cluster. Also, on a machine with 32 cores, Canvas is using around 140G of vmem. Having some way to control that would be extremely important, in my view!