search

logstash-plugins / logstash-input-jdbc

Logstash Plugin for JDBC Inputs
Apache License 2.0
449 stars 187 forks source link

High CPU usage while waiting for cron schedule #134

Open mrjameshamilton opened 8 years ago

mrjameshamilton commented 8 years ago

I noticed a high CPU usage - even though it should not be doing anything but waiting for the cron scheduled time.

In the Rufus scheduler there is a :frequency option which by default is 0.3s - could this be increased here? Maybe this would help?

jordansissel commented 7 years ago

If this is indeed the source of your cpu usage, this sounds like something we could improve in rufus. What if instead of changing the frequency, we make Rufus better about scheduling -- instead of sleep/wake/check cycles, maybe we could have it sleep exactly as long as needed until the next job?

What I am proposing is an upstream contribution.

For what it's worth, doing work every 0.3s should not have any significant impact on your cpu usage unless there is something seriously wrong.

Can you attach jstack and other profiler output for your Logstash instance so we can evaluate where the cpu usage is coming from?

jordansissel commented 7 years ago

image

The above is a screenshot with Rufus::Scheduler configured with 1 job. top -H watching only the jruby process. I am not seeing it taking much cpu.