search

adamwiggins / clockwork

A scheduler process to replace cron
662 stars 186 forks source link

SignalException: SIGTERM #10

Open Shpigford opened 13 years ago

Shpigford commented 13 years ago

I'm running clockwork on Heroku (Cedar) and I seem to randomly be getting a SignalException: SIGTERM error from clock work.

Here's the backtrace:

vendor/bundle/ruby/1.9.1/gems/clockwork-0.2.4/lib/clockwork.rb:81:in `sleep'
vendor/bundle/ruby/1.9.1/gems/clockwork-0.2.4/lib/clockwork.rb:81:in `block in run'
vendor/bundle/ruby/1.9.1/gems/clockwork-0.2.4/lib/clockwork.rb:79:in `loop'
vendor/bundle/ruby/1.9.1/gems/clockwork-0.2.4/lib/clockwork.rb:79:in `run'
lib/tasks/clockwork.rake:7:in `block (2 levels) in <top (required)>'
vendor/bundle/ruby/1.9.1/gems/rake-0.9.2/lib/rake/task.rb:205:in `call'
vendor/bundle/ruby/1.9.1/gems/rake-0.9.2/lib/rake/task.rb:205:in `block in execute'
vendor/bundle/ruby/1.9.1/gems/rake-0.9.2/lib/rake/task.rb:200:in `each'
vendor/bundle/ruby/1.9.1/gems/rake-0.9.2/lib/rake/task.rb:200:in `execute'
vendor/bundle/ruby/1.9.1/gems/rake-0.9.2/lib/rake/task.rb:158:in `block in invoke_with_call_chain'
/usr/local/lib/ruby/1.9.1/monitor.rb:201:in `mon_synchronize'
vendor/bundle/ruby/1.9.1/gems/rake-0.9.2/lib/rake/task.rb:151:in `invoke_with_call_chain'
vendor/bundle/ruby/1.9.1/gems/rake-0.9.2/lib/rake/task.rb:144:in `invoke'
vendor/bundle/ruby/1.9.1/gems/rake-0.9.2/lib/rake/application.rb:112:in `invoke_task'
vendor/bundle/ruby/1.9.1/gems/rake-0.9.2/lib/rake/application.rb:90:in `block (2 levels) in top_level'
vendor/bundle/ruby/1.9.1/gems/rake-0.9.2/lib/rake/application.rb:90:in `each'
vendor/bundle/ruby/1.9.1/gems/rake-0.9.2/lib/rake/application.rb:90:in `block in top_level'
vendor/bundle/ruby/1.9.1/gems/rake-0.9.2/lib/rake/application.rb:129:in `standard_exception_handling'
vendor/bundle/ruby/1.9.1/gems/rake-0.9.2/lib/rake/application.rb:84:in `top_level'
vendor/bundle/ruby/1.9.1/gems/rake-0.9.2/lib/rake/application.rb:62:in `block in run'
vendor/bundle/ruby/1.9.1/gems/rake-0.9.2/lib/rake/application.rb:129:in `standard_exception_handling'
vendor/bundle/ruby/1.9.1/gems/rake-0.9.2/lib/rake/application.rb:59:in `run'
vendor/bundle/ruby/1.9.1/gems/rake-0.9.2/bin/rake:32:in `<top (required)>'
vendor/bundle/ruby/1.9.1/bin/rake:19:in `load'
vendor/bundle/ruby/1.9.1/bin/rake:19:in `<main>'

And here's that rake task:

namespace :clockwork do
  desc 'Start the clockwork daemon'
  task :start => :environment do
    Clockwork.every(3.minutes, "numbers.update_all_numbers") {
      Number.delay.update_all_numbers
    }
    Clockwork.run
  end
end

Then in my Procfile I have clock: bundle exec rake clockwork:start

stevewilhelm commented 13 years ago

I am seeing the same behavior. I am also running on Heroku Cedar in a Sinatra app.

calleo commented 8 years ago

Did you guys solve this? We are seeing the same issue in Heroku. I am guessing Clockwork is not able to exit gracefully when the dyno restarts. But can it be solved somehow?

danabr75 commented 7 years ago

I'm also experiencing the same issue, on Heroku in a Rails app. Been using the gem for months, then 2 days ago, it started getting a SignalException on the sleep command. It's happened 4 times total so far.

typeoneerror commented 7 years ago

Here's what Heroku had to say

When Dynos are signal to shutdown, we first send SIGTERM, which your process should trap and perform any tasks required within 30 seconds before being forced to stop by the Dyno manager. I've had a quick look at your clock.rb, and it doesn't look like you're trapping any signals from the OS. If your application doesn't trap SIGTERM, it's likely that it will exit immediately, from whatever routine it's currently running, which looking at your stacktrace is during sleep (unsurprising for a clock process). You can gracefully shutdown by trapping the signal with something similar to:

# Trap `Kill `
Signal.trap("TERM") {
  puts "\nShutting down gracefully..."
  sleep 1
  exit
}