search

resque / resque-scheduler

A light-weight job scheduling system built on top of Resque
MIT License
1.73k stars 482 forks source link

"can't be called from trap context" still happening during shutdown post-2.1.0 #263

Closed andreacfm closed 10 years ago

andreacfm commented 11 years ago

This should have been fixed in 2.10 but after upgrading I still have this error on shutdown:

can't be called from trap context
/srv/www/getkemen.com/shared/bundle/ruby/2.0.0/gems/redis-3.0.4/lib/redis.rb:36:in `synchronize'
/srv/www/getkemen.com/shared/bundle/ruby/2.0.0/gems/redis-3.0.4/lib/redis.rb:399:in `exists'
/srv/www/getkemen.com/shared/bundle/ruby/2.0.0/gems/redis-namespace-1.3.1/lib/redis/namespace.rb:317:in `method_missing'
/srv/www/getkemen.com/shared/bundle/ruby/2.0.0/bundler/gems/resque-scheduler-bfd099376269/lib/resque_scheduler.rb:92:in `clean_schedules'
/srv/www/getkemen.com/shared/bundle/ruby/2.0.0/bundler/gems/resque-scheduler-bfd099376269/lib/resque/scheduler.rb:316:in `shutdown'
/srv/www/getkemen.com/shared/bundle/ruby/2.0.0/bundler/gems/resque-scheduler-bfd099376269/lib/resque/scheduler.rb:86:in `block in register_signal_handlers'
/srv/www/getkemen.com/shared/bundle/ruby/2.0.0/bundler/gems/resque-scheduler-bfd099376269/lib/resque/scheduler.rb:306:in `call'
/srv/www/getkemen.com/shared/bundle/ruby/2.0.0/bundler/gems/resque-scheduler-bfd099376269/lib/resque/scheduler.rb:306:in `sleep'
/srv/www/getkemen.com/shared/bundle/ruby/2.0.0/bundler/gems/resque-scheduler-bfd099376269/lib/resque/scheduler.rb:306:in `block in poll_sleep'
/srv/www/getkemen.com/shared/bundle/ruby/2.0.0/bundler/gems/resque-scheduler-bfd099376269/lib/resque/scheduler.rb:206:in `handle_shutdown'
/srv/www/getkemen.com/shared/bundle/ruby/2.0.0/bundler/gems/resque-scheduler-bfd099376269/lib/resque/scheduler.rb:306:in `poll_sleep'
/srv/www/getkemen.com/shared/bundle/ruby/2.0.0/bundler/gems/resque-scheduler-bfd099376269/lib/resque/scheduler.rb:75:in `block in run'
/srv/www/getkemen.com/shared/bundle/ruby/2.0.0/bundler/gems/resque-scheduler-bfd099376269/lib/resque/scheduler.rb:66:in `loop'
/srv/www/getkemen.com/shared/bundle/ruby/2.0.0/bundler/gems/resque-scheduler-bfd099376269/lib/resque/scheduler.rb:66:in `run'
/srv/www/getkemen.com/shared/bundle/ruby/2.0.0/bundler/gems/resque-scheduler-bfd099376269/lib/resque_scheduler/tasks.rb:29:in `block (2 levels) in <top (required)>'
meatballhat commented 11 years ago

@andreacfm Well, crap. Can you post relevant versions of resque, ruby, and resque plugins (and anything else you can think of)?

andreacfm commented 11 years ago

Rails app ( 3.2.14 ):

Thanks

meatballhat commented 11 years ago

Thanks, @andreacfm !

seamon commented 11 years ago

same exception occurred.

resque 1.24.1 resque-scheduler 2.2.0 redis 3.0.5 redis-namespace 1.3.1

what the most weird is, this exception just showed up in these days, it has been good until about one week ago. i don't remember what i did, maybe because of the "bundle update"?

meatballhat commented 11 years ago

@seamon Are you able to share your Gemfile.lock by chance?

seamon commented 11 years ago

OK. That's my Gemfile.lock, please check it out. Thank you so much.

GIT
  remote: git://github.com/FriendlyId/friendly_id.git
  revision: 2764077e3af32a88114d68c75e20ef738e4fa829
  specs:
    friendly_id (5.0.0)
      activerecord (~> 4.0.0)

GIT
  remote: git://github.com/freebird0221/simple-captcha.git
  revision: a4fabb3e847ada638df2c4c990b39e1706c17520
  specs:
    simple_captcha (0.1.5)

GIT
  remote: git://github.com/plataformatec/simple_form.git
  revision: bc97459dc87f944503cd7c96f02b98b432816b8d
  specs:
    simple_form (3.0.0)
      actionpack (>= 4.0.0, < 4.1)
      activemodel (>= 4.0.0, < 4.1)

GIT
  remote: git://github.com/tsmango/default_value_for.git
  revision: 52f1c40f0aa1a043ae15a69379a08ab0a5421c6b
  specs:
    default_value_for (2.0.2)

GEM
  remote: http://ruby.taobao.org/
  specs:
    actionmailer (4.0.0)
      actionpack (= 4.0.0)
      mail (~> 2.5.3)
    actionpack (4.0.0)
      activesupport (= 4.0.0)
      builder (~> 3.1.0)
      erubis (~> 2.7.0)
      rack (~> 1.5.2)
      rack-test (~> 0.6.2)
    active_hash (1.0.2)
      activesupport (>= 2.2.2)
    activemodel (4.0.0)
      activesupport (= 4.0.0)
      builder (~> 3.1.0)
    activerecord (4.0.0)
      activemodel (= 4.0.0)
      activerecord-deprecated_finders (~> 1.0.2)
      activesupport (= 4.0.0)
      arel (~> 4.0.0)
    activerecord-deprecated_finders (1.0.3)
    activesupport (4.0.0)
      i18n (~> 0.6, >= 0.6.4)
      minitest (~> 4.2)
      multi_json (~> 1.3)
      thread_safe (~> 0.1)
      tzinfo (~> 0.3.37)
    acts_as_follower (0.2.0)
    acts_as_list (0.3.0)
      activerecord (>= 3.0)
    ancestry (2.0.0)
      activerecord (>= 3.0.0)
    animate-rails (1.0.1)
      rails
    anjlab-bootstrap-rails (3.0.0.0)
      railties (>= 3.0)
      sass (>= 3.2)
    ansi (1.4.3)
    arel (4.0.0)
    atomic (1.1.14)
    bcrypt-ruby (3.1.2)
    better_errors (1.0.1)
      coderay (>= 1.0.0)
      erubis (>= 2.6.6)
    binding_of_caller (0.7.2)
      debug_inspector (>= 0.0.1)
    bluepill (0.0.64)
      activesupport (>= 3.0.0)
      daemons (~> 1.1.4)
      i18n (>= 0.5.0)
      state_machine (~> 1.1)
    builder (3.1.4)
    cancan (1.6.10)
    capistrano (3.0.0)
      i18n
      rake (>= 10.0.0)
      sshkit (>= 0.0.23)
    capistrano-bundler (1.0.0)
      capistrano (>= 3.0.0.pre)
    capistrano-rails (1.0.0)
      capistrano
    carrierwave (0.9.0)
      activemodel (>= 3.2.0)
      activesupport (>= 3.2.0)
      json (>= 1.7)
    cells (3.8.8)
      actionpack (>= 3.0)
      railties (>= 3.0)
    chinese_pinyin (0.4.2)
    climate_control (0.0.3)
      activesupport (>= 3.0)
    cocaine (0.5.2)
      climate_control (>= 0.0.3, < 1.0)
    coderay (1.0.9)
    coffee-rails (4.0.0)
      coffee-script (>= 2.2.0)
      railties (>= 4.0.0.beta, < 5.0)
    coffee-script (2.2.0)
      coffee-script-source
      execjs
    coffee-script-source (1.6.3)
    daemons (1.1.9)
    debug_inspector (0.0.2)
    devise (3.1.1)
      bcrypt-ruby (~> 3.0)
      orm_adapter (~> 0.1)
      railties (>= 3.2.6, < 5)
      thread_safe (~> 0.1)
      warden (~> 1.2.3)
    devise-async (0.8.0)
      devise (>= 2.2, < 3.2)
    diff-lcs (1.2.4)
    domain_name (0.5.14)
      unf (>= 0.0.5, < 1.0.0)
    erubis (2.7.0)
    ethon (0.6.1)
      ffi (>= 1.3.0)
      mime-types (~> 1.18)
    eventmachine (1.0.3)
    execjs (2.0.2)
    factory_girl (4.2.0)
      activesupport (>= 3.0.0)
    factory_girl_rails (4.2.1)
      factory_girl (~> 4.2.0)
      railties (>= 3.0.0)
    faraday (0.8.8)
      multipart-post (~> 1.2.0)
    ffi (1.9.0)
    font-awesome-sass-rails (3.0.2.2)
      railties (>= 3.1.1)
      sass-rails (>= 3.1.1)
    hashie (2.0.5)
    hashr (0.0.22)
    hike (1.2.3)
    hpricot (0.8.6)
    http-cookie (1.0.2)
      domain_name (~> 0.5)
    httpauth (0.2.0)
    i18n (0.6.5)
    i18n_generators (1.2.1)
      mechanize
      rails (>= 3.0.0)
    jbuilder (1.5.2)
      activesupport (>= 3.0.0)
      multi_json (>= 1.2.0)
    jquery-rails (2.2.1)
      railties (>= 3.0, < 5.0)
      thor (>= 0.14, < 2.0)
    jquery-ui-rails (4.0.5)
      railties (>= 3.1.0)
    json (1.8.0)
    jwt (0.1.8)
      multi_json (>= 1.5)
    kgio (2.8.1)
    libv8 (3.16.14.3)
    mail (2.5.4)
      mime-types (~> 1.16)
      treetop (~> 1.4.8)
    mechanize (2.7.2)
      domain_name (~> 0.5, >= 0.5.1)
      http-cookie (~> 1.0.0)
      mime-types (~> 1.17, >= 1.17.2)
      net-http-digest_auth (~> 1.1, >= 1.1.1)
      net-http-persistent (~> 2.5, >= 2.5.2)
      nokogiri (~> 1.4)
      ntlm-http (~> 0.1, >= 0.1.1)
      webrobots (>= 0.0.9, < 0.2)
    method_source (0.8.2)
    mime-types (1.25)
    mini_magick (3.6.0)
      subexec (~> 0.2.1)
    mini_portile (0.5.1)
    minitest (4.7.5)
    mono_logger (1.1.0)
    multi_json (1.8.2)
    multipart-post (1.2.0)
    mysql2 (0.3.13)
    net-http-digest_auth (1.4)
    net-http-persistent (2.9)
    net-scp (1.1.2)
      net-ssh (>= 2.6.5)
    net-ssh (2.7.0)
    newrelic_rpm (3.6.8.164)
    nokogiri (1.6.0)
      mini_portile (~> 0.5.0)
    ntlm-http (0.1.1)
    oauth2 (0.8.1)
      faraday (~> 0.8)
      httpauth (~> 0.1)
      jwt (~> 0.1.4)
      multi_json (~> 1.0)
      rack (~> 1.2)
    omniauth (1.1.4)
      hashie (>= 1.2, < 3)
      rack
    omniauth-oauth2 (1.1.1)
      oauth2 (~> 0.8.0)
      omniauth (~> 1.0)
    omniauth-qq-connect (0.2.0)
      omniauth (~> 1.0)
      omniauth-oauth2 (~> 1.0)
    omniauth-renren-oauth2 (0.0.6)
      omniauth (~> 1.1)
      omniauth-oauth2 (~> 1.1)
    omniauth-weibo-oauth2 (0.3.0)
      omniauth (~> 1.0)
      omniauth-oauth2 (~> 1.0)
    orm_adapter (0.4.0)
    paperclip (3.5.1)
      activemodel (>= 3.0.0)
      activesupport (>= 3.0.0)
      cocaine (~> 0.5.0)
      mime-types
    polyamorous (0.6.4)
      activerecord (>= 3.0)
    polyglot (0.3.3)
    protected_attributes (1.0.3)
      activemodel (>= 4.0.0, < 5.0)
    pry (0.9.12.2)
      coderay (~> 1.0.5)
      method_source (~> 0.8)
      slop (~> 3.4)
    pry-nav (0.2.3)
      pry (~> 0.9.10)
    pry-rails (0.3.2)
      pry (>= 0.9.10)
    quiet_assets (1.0.2)
      railties (>= 3.1, < 5.0)
    rack (1.5.2)
    rack-protection (1.5.0)
      rack
    rack-test (0.6.2)
      rack (>= 1.0)
    rails (4.0.0)
      actionmailer (= 4.0.0)
      actionpack (= 4.0.0)
      activerecord (= 4.0.0)
      activesupport (= 4.0.0)
      bundler (>= 1.3.0, < 2.0)
      railties (= 4.0.0)
      sprockets-rails (~> 2.0.0)
    rails-observers (0.1.2)
      activemodel (~> 4.0)
    rails_kindeditor (0.4.0)
      carrierwave
      mini_magick
    railties (4.0.0)
      actionpack (= 4.0.0)
      activesupport (= 4.0.0)
      rake (>= 0.8.7)
      thor (>= 0.18.1, < 2.0)
    rainbows (4.5.0)
      kgio (~> 2.5)
      rack (~> 1.1)
      unicorn (~> 4.6, >= 4.6.2)
    raindrops (0.12.0)
    rake (10.1.0)
    ransack (1.0.0)
      actionpack (>= 3.0)
      activerecord (>= 3.0)
      polyamorous (~> 0.6.0)
    rdoc (3.12.2)
      json (~> 1.4)
    redis (3.0.5)
    redis-activesupport (4.0.0)
      activesupport (~> 4)
      redis-store (~> 1.1.0)
    redis-namespace (1.3.1)
      redis (~> 3.0.0)
    redis-objects (0.7.0)
      redis (>= 3.0.2)
    redis-store (1.1.4)
      redis (>= 2.2)
    ref (1.0.5)
    resque (1.24.1)
      mono_logger (~> 1.0)
      multi_json (~> 1.0)
      redis-namespace (~> 1.2)
      sinatra (>= 0.9.2)
      vegas (~> 0.1.2)
    resque-scheduler (2.2.0)
      redis (>= 3.0.0)
      resque (>= 1.20.0, < 1.25)
      rufus-scheduler (~> 2.0)
    rest-client (1.6.7)
      mime-types (>= 1.16)
    rspec-core (2.14.6)
    rspec-expectations (2.14.3)
      diff-lcs (>= 1.1.3, < 2.0)
    rspec-mocks (2.14.4)
    rspec-rails (2.14.0)
      actionpack (>= 3.0)
      activesupport (>= 3.0)
      railties (>= 3.0)
      rspec-core (~> 2.14.0)
      rspec-expectations (~> 2.14.0)
      rspec-mocks (~> 2.14.0)
    ruby-ole (1.2.11.7)
    ruby_regex (0.1.0)
    rufus-scheduler (2.0.24)
      tzinfo (>= 0.3.22)
    sass (3.2.12)
    sass-rails (4.0.1)
      railties (>= 4.0.0, < 5.0)
      sass (>= 3.1.10)
      sprockets-rails (~> 2.0.0)
    sdoc (0.3.20)
      json (>= 1.1.3)
      rdoc (~> 3.10)
    settingslogic (2.0.9)
    sinatra (1.4.3)
      rack (~> 1.4)
      rack-protection (~> 1.4)
      tilt (~> 1.3, >= 1.3.4)
    slop (3.4.6)
    spreadsheet (0.9.0)
      ruby-ole (>= 1.0)
    sprockets (2.10.0)
      hike (~> 1.2)
      multi_json (~> 1.0)
      rack (~> 1.0)
      tilt (~> 1.1, != 1.3.0)
    sprockets-rails (2.0.0)
      actionpack (>= 3.0)
      activesupport (>= 3.0)
      sprockets (~> 2.8)
    sshkit (0.0.34)
      net-scp
      net-ssh
      term-ansicolor
    state_machine (1.2.0)
    subexec (0.2.3)
    symbolize (4.4.1)
      activemodel (>= 3.2, < 5)
      activesupport (>= 3.2, < 5)
    term-ansicolor (1.2.2)
      tins (~> 0.8)
    therubyracer (0.12.0)
      libv8 (~> 3.16.14.0)
      ref
    thor (0.18.1)
    thread_safe (0.1.3)
      atomic
    tilt (1.4.1)
    timecop (0.6.3)
    tins (0.11.0)
    tire (0.6.1)
      activemodel (>= 3.0)
      activesupport
      ansi
      hashr (~> 0.0.19)
      multi_json (~> 1.3)
      rake
      rest-client (~> 1.6)
    treetop (1.4.15)
      polyglot
      polyglot (>= 0.3.1)
    turbolinks (1.3.0)
      coffee-rails
    typhoeus (0.6.5)
      ethon (~> 0.6.1)
    tzinfo (0.3.38)
    uglifier (2.2.1)
      execjs (>= 0.3.0)
      multi_json (~> 1.0, >= 1.0.2)
    unf (0.1.2)
      unf_ext
    unf_ext (0.0.6)
    unicorn (4.6.3)
      kgio (~> 2.6)
      rack
      raindrops (~> 0.7)
    vegas (0.1.11)
      rack (>= 1.0.0)
    warden (1.2.3)
      rack (>= 1.0)
    webrobots (0.1.1)
    will_paginate (3.0.5)
    yajl-ruby (1.1.0)

PLATFORMS
  ruby

DEPENDENCIES
  active_hash (~> 1.0.2)
  acts_as_follower
  acts_as_list
  ancestry
  animate-rails
  anjlab-bootstrap-rails (= 3.0.0.0)
  bcrypt-ruby
  better_errors
  binding_of_caller
  bluepill
  cancan
  capistrano
  capistrano-bundler
  capistrano-rails
  carrierwave
  cells
  chinese_pinyin (~> 0.4.2)
  coffee-rails
  default_value_for!
  devise
  devise-async
  eventmachine
  execjs
  factory_girl_rails
  font-awesome-sass-rails
  friendly_id!
  hpricot
  i18n_generators
  jbuilder
  jquery-rails (= 2.2.1)
  jquery-ui-rails
  mechanize
  mini_magick
  mysql2
  newrelic_rpm
  nokogiri
  omniauth
  omniauth-qq-connect
  omniauth-renren-oauth2
  omniauth-weibo-oauth2
  paperclip
  protected_attributes
  pry-nav
  pry-rails
  quiet_assets
  rails
  rails-observers
  rails_kindeditor
  rainbows
  ransack
  redis-activesupport
  redis-objects
  resque
  resque-scheduler
  rspec-rails
  ruby_regex
  sass-rails
  sdoc
  settingslogic
  simple_captcha!
  simple_form!
  spreadsheet
  sshkit
  state_machine
  symbolize
  therubyracer
  timecop
  tire
  turbolinks
  typhoeus
  uglifier
  unicorn
  will_paginate
  yajl-ruby
meatballhat commented 11 years ago

@seamon Thanks!

saimonmoore commented 10 years ago

@meatballhat just ran into this. Any progress here?

meatballhat commented 10 years ago

@saimonmoore no, sorry! As always, any help is greatly appreciated, but I'm assuming you've already done a decent amount of digging.

saimonmoore commented 10 years ago

I am digging into it. I need this fixed asap for a production release. On 15 Nov 2013 03:34, "Dan Buch" notifications@github.com wrote:

@saimonmoore https://github.com/saimonmoore no, sorry! As always, any help is greatly appreciated, but I'm assuming you've already done a decent amount of digging.

— Reply to this email directly or view it on GitHubhttps://github.com/resque/resque-scheduler/issues/263#issuecomment-28542972 .

sgirones commented 10 years ago

Relevant info here: https://bugs.ruby-lang.org/issues/7917

It's simply Ruby 2 doesn't allow "deadlock-like" code inside a trap. Ruby 1.9.3 do so. I guess the shutdown process should be reimplemented in order to avoid the synchronize inside ../lib/redis.rb:36

saimonmoore commented 10 years ago

@sgirones :clap:

sgirones commented 10 years ago

Foreman guys ran into this same situation, and here is their solution: https://github.com/ddollar/foreman/issues/332

I don't think I'm able to implement something like that right now. Don't know enough about resque-scheduler internals.

meatballhat commented 10 years ago

@sgirones :clap: indeed! @saimonmoore thanks for your patience on this. I'll be stealing from the Foreman folks shortly.

ksinkar commented 10 years ago

@meatballhat I think it is safe to close this issue.

sgirones commented 10 years ago

good job guys!