ArgumentError: A copy of MyTestJob has beed removed from the module tree but is still active!

[rwz]

Hi,

I'm trying to run SuckerPunch in development mode and run into a problem when Rails reloads classes between requests, but Queue has a class instance and tries to use it resulting with subj error.

Is there a way around this? I don't want to switch to inline mode for development.

[brandonhilkert]

I haven't experienced that. Can you show me some code?

[rwz]

Ruby 2.1.1 Rails 4.0.3 Job:

class SendInvitationJob
  include SuckerPunch::Job

  def perform(email, term)
    with_ar_connection do
      invitation = Invitation.create!(email: email, term: term)
      UserMailer.invitation(invitation).deliver
    end
  end

  private

  def with_ar_connection(&block)
    ActiveRecord::Base.connection_pool.with_connection(&block)
  end
end

controller:

class MyController < ApplicationController
  def create
    [1,2,3].each do |num|
      SendInvitationJob.new.async.perform("joe#{num}@example.com", "hello")
    end
  end
end

Log:

Celluloid::PoolManager: async call `perform` aborted!
ArgumentError: A copy of SendInvitationJob has been removed from the module tree but is still active!
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/activesupport-4.0.3/lib/active_support/dependencies.rb:446:in `load_missing_constant'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/activesupport-4.0.3/lib/active_support/dependencies.rb:184:in `const_missing'
        /Users/dev/my-app/app/jobs/send_invitation_job.rb:6:in `block in perform'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/activerecord-4.0.3/lib/active_record/connection_adapters/abstract/connection_pool.rb:294:in `with_connection'
        /Users/dev/my-app/app/jobs/send_invitation_job.rb:14:in `with_ar_connection'
        /Users/dev/my-app/app/jobs/send_invitation_job.rb:5:in `perform'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/calls.rb:25:in `public_send'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/calls.rb:25:in `dispatch'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/calls.rb:67:in `dispatch'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/actor.rb:322:in `block in handle_message'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/actor.rb:416:in `block in task'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/tasks.rb:55:in `block in initialize'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/tasks/task_fiber.rb:13:in `block in create'
        (celluloid):0:in `remote procedure call'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/calls.rb:92:in `value'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/proxies/sync_proxy.rb:33:in `method_missing'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/proxies/actor_proxy.rb:20:in `_send_'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/pool_manager.rb:41:in `_send_'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/pool_manager.rb:123:in `method_missing'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/calls.rb:25:in `public_send'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/calls.rb:25:in `dispatch'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/calls.rb:122:in `dispatch'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/actor.rb:322:in `block in handle_message'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/actor.rb:416:in `block in task'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/tasks.rb:55:in `block in initialize'
        /Users/dev/.rvm/gems/ruby-2.1.1@my-app/gems/celluloid-0.15.2/lib/celluloid/tasks/task_fiber.rb:13:in `block in create'

[brandonhilkert]

I see....I'll need to play with this. I've not experienced this, but also can't say that my dev environment runs many of these tasks that are at all long running. Not that your case it, but sending 3 emails could take a bit. Have any ideas off hand?

[rwz]

Yep, I thought this is a known problem and already has a solution that I missed somehow.

So, I could actually try to fix that and send a pull request later. Maybe this weekend.

The idea is to use string with class name instead of actual class instance in the Queue and do klass_name.constantize every time you want to use it in your worker.

Also, this behavior only makes sense for Rails environment when Rails.application.config.cache_classes is false.

[brandonhilkert]

The name is currently saved to the celluloid registry with the class name underscored

    def name
      klass.to_s.underscore.to_sym
    end

The celluloid pool is saved to the registry. Now that I look at the log output, it looks like the job was still running, but it the Pool now has an outdated reference to the class. I'm not sure it makes sense to new up a new pool each time. I'd have to think about how to avoid that in production so the queue would continue between each invocation.

[concept47]

I'm having this problem trying to run a SuckerPunch Job in development from the Rails Console. Runs well the first time, but after I do a

reload!

I get

MailerJob crashed!
ArgumentError: A copy of MailerJob has been removed from the module tree but is still active!

I'm just doing this at the command line

MailerJob.new.async.welcome(1)

my job code is

class MailerJob
  include SuckerPunch::Job

   def welcome(user_id)
     user = User.find(user_id)
     WelcomeMailer.welcome(user).deliver
  end

[divineslight]

I am having same issue in Rails 4.0.2 dev environment:

MailJob crashed!
ArgumentError: A copy of MailJob has been removed from the module tree but is still active!
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/activesupport-4.0.2/lib/active_support    /dependencies.rb:446:in `load_missing_constant'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/activesupport-4.0.2/lib/active_support/dependencies.rb:184:in `const_missing'
/home/black/work/ttest/app/jobs/mail_job.rb:6:in `perform'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/calls.rb:25:in `public_send'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/calls.rb:25:in `dispatch'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/calls.rb:67:in `dispatch'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/actor.rb:322:in `block in handle_message'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/actor.rb:416:in `block in task'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/tasks.rb:55:in `block in initialize'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/tasks/task_fiber.rb:13:in `block in create'
Celluloid::PoolManager: async call `perform` aborted!
ArgumentError: A copy of MailJob has been removed from the module tree but is still active!
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/activesupport-4.0.2/lib/active_support/dependencies.rb:446:in `load_missing_constant'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/activesupport-4.0.2/lib/active_support/dependencies.rb:184:in `const_missing'
/home/black/work/ttest/app/jobs/mail_job.rb:6:in `perform'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/calls.rb:25:in `public_send'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/calls.rb:25:in `dispatch'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/calls.rb:67:in `dispatch'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/actor.rb:322:in `block in handle_message'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/actor.rb:416:in `block in task'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/tasks.rb:55:in `block in initialize'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/tasks/task_fiber.rb:13:in `block in create'
(celluloid):0:in `remote procedure call'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/calls.rb:92:in `value'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/proxies/sync_proxy.rb:33:in `method_missing'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/proxies/actor_proxy.rb:20:in `_send_'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/pool_manager.rb:41:in `_send_'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/pool_manager.rb:123:in `method_missing'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/calls.rb:25:in `public_send'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/calls.rb:25:in `dispatch'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/calls.rb:122:in `dispatch'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/actor.rb:322:in `block in handle_message'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/actor.rb:416:in `block in task'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/tasks.rb:55:in `block in initialize'
/home/black/.rvm/gems/ruby-2.0.0-p353@loop/gems/celluloid-0.15.2/lib/celluloid/tasks/task_fiber.rb:13:in `block in create'

[brandonhilkert]

Give version 1.0.3 a shot.

[mvdamme]

I'm experiencing the same problem on a Rails 4.1.12 app. I was originally using sucker_punch 1.0.5, but upgrading to the latest released version (1.5.1) didn't help.

If I add a sleep call to the 'main' thread (the one creating the job) so the job finishes before the request (which redirects and triggers a subsequent reload) the problem disappears.

Any suggestions?

[nynhex]

@brandonhilkert I'm seeing this same problem both in development and in productions. Rails 4.2.4, Ruby 2.2.3. My job sends a twilio message and calls a mailer each one taking a second or so to perform. Should I break these into two separate jobs and have them fire in parallel?

[brandonhilkert]

Something like http://ruby-concurrency.github.io/concurrent-ruby/Concurrent/CountDownLatch.html is a great way to detect whether the jobs in the other threads have finished.