Open swlong opened 5 years ago
Hi Wesley,
Did you have figure out what caused this? I'm noticing something similar.
If Pilon is unexpectedly using multiple threads, it makes it harder to be a "good neighbor" when using shared resources.
Best,
Dave
Dave,
Sorry, I never got any feedback on this from Broad, and we never determined the precise cause. I will say that I suspect it may be something related to Java, because we have another bioinformatics program (also a jar) and it exhibits a very similar behavior where it is spiking and using more processors than it is supposed to be using on the same server. So it may be something about java or how it is configured on our cluster and Torque PBS, but no one has determined the precise cause or how to correct or mitigate it.
Best of luck, S. Wesley Long
Sorry for not responding to this way back when. This is when the java garbage collector runs to try to free up memory.
My cluster administrator has asked me to look into why pilon (v1.22) apparently spikes its cpu utilization and grabs as many as 31 cores for a few minutes, when I've limited the overall threads per job to 4 (due to cluster restrictions).
There's a "--threads" option for pilon which is defaulted to 1 and "experimental." To be clear, I'm not setting this option so it should be defaulted to 1. I'm at a loss to explain the behavior to the admin.
Appreciate any help.
Best, S. Wesley Long