search

rails / activejob

Declare job classes that can be run by a variety of queueing backends
743 stars 47 forks source link

ActiveJob with multiple databases #112

Closed tetherit closed 9 years ago

tetherit commented 9 years ago

Hi there,

I have multiple databases that are switched to based on subdomain in my application controller, using:

Mongoid.override_database(database_name)

In my controller, I am running:

UserMailer.password_reset(user, timeline_name).deliver_later

But I am getting the following exception, which is understandable, as I cannot see a way to let ActiveJob know which database to use:

2015-09-03T13:48:31.159Z 11467 TID-18to6c WARN: /var/lib/gems/2.2.0/gems/mongoid-5.0.0.rc0/lib/mongoid/criteria.rb:508:in `check_for_missing_documents!'
/var/lib/gems/2.2.0/gems/mongoid-5.0.0.rc0/lib/mongoid/criteria/findable.rb:20:in `execute_or_raise'
/var/lib/gems/2.2.0/gems/mongoid-5.0.0.rc0/lib/mongoid/criteria/findable.rb:40:in `find'
/var/lib/gems/2.2.0/gems/mongoid-5.0.0.rc0/lib/mongoid/findable.rb:93:in `find'
/var/lib/gems/2.2.0/gems/globalid-0.3.6/lib/global_id/locator.rb:132:in `locate'
/var/lib/gems/2.2.0/gems/globalid-0.3.6/lib/global_id/locator.rb:158:in `block in locate'
/var/lib/gems/2.2.0/gems/mongoid-5.0.0.rc0/lib/mongoid/scopable.rb:193:in `block in unscoped'
/var/lib/gems/2.2.0/gems/mongoid-5.0.0.rc0/lib/mongoid/scopable.rb:245:in `without_default_scope'
/var/lib/gems/2.2.0/gems/mongoid-5.0.0.rc0/lib/mongoid/scopable.rb:192:in `unscoped'
/var/lib/gems/2.2.0/gems/globalid-0.3.6/lib/global_id/locator.rb:158:in `locate'
/var/lib/gems/2.2.0/gems/globalid-0.3.6/lib/global_id/locator.rb:17:in `locate'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/arguments.rb:97:in `deserialize_global_id'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/arguments.rb:83:in `deserialize_argument'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/arguments.rb:40:in `block in deserialize'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/arguments.rb:40:in `map'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/arguments.rb:40:in `deserialize'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/core.rb:90:in `deserialize_arguments'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/core.rb:80:in `deserialize_arguments_if_needed'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/execution.rb:30:in `perform_now'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/execution.rb:21:in `execute'
/var/lib/gems/2.2.0/gems/activejob-4.2.4/lib/active_job/queue_adapters/sidekiq_adapter.rb:42:in `perform'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/processor.rb:75:in `execute_job'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/processor.rb:52:in `block (2 levels) in process'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/chain.rb:127:in `block in invoke'
/var/lib/gems/2.2.0/gems/newrelic_rpm-3.13.0.299/lib/new_relic/agent/instrumentation/sidekiq.rb:33:in `block in call'
/var/lib/gems/2.2.0/gems/newrelic_rpm-3.13.0.299/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:362:in `perform_action_with_newrelic_trace'
/var/lib/gems/2.2.0/gems/newrelic_rpm-3.13.0.299/lib/new_relic/agent/instrumentation/sidekiq.rb:29:in `call'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
/var/lib/gems/2.2.0/gems/sidekiq-unique-jobs-3.0.14/lib/sidekiq_unique_jobs/middleware/server/unique_jobs.rb:16:in `call'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/server/retry_jobs.rb:74:in `call'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/server/logging.rb:11:in `block in call'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/logging.rb:30:in `with_context'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/server/logging.rb:7:in `call'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/chain.rb:132:in `call'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/middleware/chain.rb:132:in `invoke'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/processor.rb:51:in `block in process'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/processor.rb:98:in `stats'
/var/lib/gems/2.2.0/gems/sidekiq-3.5.0/lib/sidekiq/processor.rb:50:in `process'
/var/lib/gems/2.2.0/gems/celluloid-0.17.1.2/lib/celluloid/calls.rb:28:in `public_send'
/var/lib/gems/2.2.0/gems/celluloid-0.17.1.2/lib/celluloid/calls.rb:28:in `dispatch'
/var/lib/gems/2.2.0/gems/celluloid-0.17.1.2/lib/celluloid/call/async.rb:7:in `dispatch'
/var/lib/gems/2.2.0/gems/celluloid-0.17.1.2/lib/celluloid/cell.rb:50:in `block in dispatch'
/var/lib/gems/2.2.0/gems/celluloid-0.17.1.2/lib/celluloid/cell.rb:76:in `block in task'
/var/lib/gems/2.2.0/gems/celluloid-0.17.1.2/lib/celluloid/actor.rb:339:in `block in task'
/var/lib/gems/2.2.0/gems/celluloid-0.17.1.2/lib/celluloid/task.rb:44:in `block in initialize'
/var/lib/gems/2.2.0/gems/celluloid-0.17.1.2/lib/celluloid/task/fibered.rb:14:in `block in create'
2015-09-03T13:48:52.729Z 11467 TID-18ft2k ActionMailer::DeliveryJob JID-309e70aff24990115bb17718 INFO: start
2015-09-03T13:48:52.738Z 11467 TID-18ft2k ActionMailer::DeliveryJob JID-309e70aff24990115bb17718 INFO: fail: 0.009 sec
2015-09-03T13:48:52.741Z 11467 TID-18ft2k WARN: {"class"=>"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper", "wrapped"=>"ActionMailer::DeliveryJob", "queue"=>"mailers", "args"=>[{"job_class"=>"ActionMailer::DeliveryJob", "job_id"=>"6f95a56e-4a39-424d-bfc6-82a474774cfb", "queue_name"=>"mailers", "arguments"=>["UserMailer", "password_reset", "deliver_now", {"_aj_globalid"=>"gid://timeline/User/55230a4278616e6fbc010000"}, "monitor"], "locale"=>"en"}], "retry"=>true, "jid"=>"309e70aff24990115bb17718", "created_at"=>1441287513.5178096, "enqueued_at"=>1441288132.7244759, "error_message"=>"Error while trying to deserialize arguments: \nmessage:\n  Document(s) not found for class User with id(s) 55230a4278616e6fbc010000.\nsummary:\n  When calling User.find with an id or array of ids, each parameter must match a document in the database or this error will be raised. The search was for the id(s): 55230a4278616e6fbc010000 ... (1 total) and the following ids were not found: 55230a4278616e6fbc010000.\nresolution:\n  Search for an id that is in the database or set the Mongoid.raise_not_found_error configuration option to false, which will cause a nil to be returned instead of raising this error when searching for a single id, or only the matched documents when searching for multiples.", "error_class"=>"ActiveJob::DeserializationError", "failed_at"=>1441288079.3246949, "retry_count"=>1, "retried_at"=>1441288132.7376645}
2015-09-03T13:48:52.741Z 11467 TID-18ft2k WARN: ActiveJob::DeserializationError: Error while trying to deserialize arguments: 
message:
  Document(s) not found for class User with id(s) 55230a4278616e6fbc010000.
summary:
  When calling User.find with an id or array of ids, each parameter must match a document in the database or this error will be raised. The search was for the id(s): 55230a4278616e6fbc010000 ... (1 total) and the following ids were not found: 55230a4278616e6fbc010000.
resolution:
  Search for an id that is in the database or set the Mongoid.raise_not_found_error configuration option to false, which will cause a nil to be returned instead of raising this error when searching for a single id, or only the matched documents when searching for multiples.

I can see a way to specify a queue, but how do I specify a database?

cristianbica commented 9 years ago

There's isn't an easy way do do that. The error is triggered by GlobalID deserialization and there's no way to hook before that. I think the way to do this is using GlobalID finders

tetherit commented 9 years ago

I can see how to use a custom app locator which I think I will need to do, but I cannot see a way to add the database name (or subdomain) to the global id string. Any ideas?

cristianbica commented 9 years ago

see my answer in GlobalID issue. Closing this as this is a GlobalID discussion

tetherit commented 9 years ago

Makes sense - your solution at https://github.com/rails/globalid/issues/78 is a good one, trying it now!