search

tseemann / barrnap

:microscope: :leo: Bacterial ribosomal RNA predictor
GNU General Public License v3.0
210 stars 41 forks source link

Default # of cores leads to OOM reaping on Linux #21

Closed taltman closed 6 years ago

taltman commented 7 years ago

First off, thanks for writing this useful tool!

Today, I started noticing that my barrnap jobs were dying with the following message:

[12:58:10] bad line in nhmmer output - Killed

I then ran nhmmer directly, and noticed that the barrnap script was likely not happy with the 'Killed' text at the bottom of the nhmmer output.

Looking at top, I noticed that the nhmmer process was spawning threads which triggered the OOM-Killer on my Linux machine. I didn't notice until now that barrnap defaults to 8 threads. That might be reasonable on a beefy server, but on my Linux instance, I only had 2 CPUs and 2 GB of RAM.

I propose that barrnap should default to using a single CPU unless the user explicitly overrides it. Otherwise, barrnap is violating the Law of Least Surprise. Or, barrnap should be sure not to exceed the number of CPUs on the system, as that is surely undesirable behavior. Some bioinformatic tools that I've worked with will default to using (N-K) CPUs or one CPU, whichever is larger (where N is the total number of CPUs, and K in {1,2}, as a buffer to prevent the machine from locking up).

In the mean-time, I am now making sure to set the threads argument explicitly. Thanks for your consideration!

tseemann commented 6 years ago

Fixed in https://github.com/tseemann/barrnap/commit/fbeda77a40c6b302cb60c92ce29b0a65519f303a with new release soon.