search

TheJacksonLaboratory / LIRICAL

LIkelihood Ratio Interpretation of Clinical AbnormaLities
https://thejacksonlaboratory.github.io/LIRICAL/stable
Other
24 stars 11 forks source link

Threading problem #559

Closed seoanezonjic closed 1 year ago

seoanezonjic commented 2 years ago

When I use Lirical in phenopacket mode it uses an arbitrary number of processors. I would like to set the exact number of cpus because I execute this tool in a shared computing cluster and I have to define the resources that would be used. Thanks in advance

ielis commented 2 years ago

Hello, @seoanezonjic , sorry for a late response. The number of processors to use is not easy to control from LIRICAL. However, this post outlines how to use -XX:ActiveProcessorCount option to limit the number of CPUs that are available to a VM. Can you give it try and let me know if it helped?

seoanezonjic commented 2 years ago

Hi @ielis The option works but when I check the resource consumption with top, the cpu % is about a 100-150 above the indicated cpu number. What is the reason of this? Thank you by your help

ielis commented 2 years ago

Hi @seoanezonjic , in general, Java programmer does not have absolute control over the number of cores or threads used by the app. There is some control, but not absolute control. I see two main reasons why we cannot guarantee specific CPU usage. First, we use a library for defining gene and transcript locations. Deserialization uses several threads to speed up the process, but the library API does not expose control over the thread count. In general, we cannot limit the CPU usage in libraries. This should be covered by -XX:ActiveProcessorCount. Second, running a JVM creates an overhead due to e.g. garbage collection. So, if the app runs on 100%, there is JVM overhead, you can see higher CPU usage. This is, again, outside of LIRICAL scope.

If the issue is impeding your work, you've got to ask the admins of your compute environment for help. I'm not sure we can do more than -XX:ActiveProcessorCount.