rosa
commented
1 month ago Ohhh, totally! Let me look into this one to see how to fix, I think it's something Solid Queue should handle. Thanks for the report!
Open bubiche opened 1 month ago
rosa
commented
1 month ago Ohhh, totally! Let me look into this one to see how to fix, I think it's something Solid Queue should handle. Thanks for the report!
djpate
commented
1 month ago Same issue here. is there a workaround? my entire controller is in a connected_to block?
I tried wrapping the perform_later in another connection switch but no luck.
ActiveRecord::Base.connected_to(role: :writing, shard: :queue) do
Workers::Foobar.perform_later(account_id: account_id, id: id)
end
thibaudgg
commented
2 weeks ago Same problem for me, I'm working on switching my multi-tenant app to SQLite (one database per tenant) using sharding, while keep the SolidQueue database separated.
Please let me know how I could help, or if sharing more code would be helpful. I'm happy to help fix the issue as well.
class ApplicationRecord < ActiveRecord::Base
self.abstract_class = true
connects_to shards: Tenant.shards.map { |shard|
[ shard, { writing: shard } ]
}.to_h
endProduction config:
config.active_job.queue_adapter = :solid_queue
config.solid_queue.connects_to = { database: { writing: :queue } }Tenant switching logic (simplified):
module Tenant
extend self
def switch(tenant)
ActiveRecord::Base.connected_to(shard: tenant.to_sym) do
yield
end
end
# ...
endBut trying to enqueue jobs gives me that error trace:
thibaudgg
commented
2 weeks ago Added a comment on this other ticket (https://github.com/rails/solid_queue/issues/353), that might help here.
rosa
commented
2 weeks ago Sorry for the delay on this one, I haven't managed to look into it yet, but it's definitely on my radar. I'll try next week 🤞
thibaudgg
commented
2 weeks ago Thanks @rosa!
I just gave a tried to this patch on production, and it seems to work like a charm.
# config/initializers/solid_queue_patch.rb
Rails.application.config.to_prepare do
module CurrentShardPatch
def current_shard; :queue end
end
SolidQueue::Record.send(:extend, CurrentShardPatch)
endMy app is open-source, so you can give it a look if that's helpful to you. Here some relevant parts:
rosa
commented
2 weeks ago Oh, awesome! I'm sure this is going to be super helpful! I'll look into all those next week, too 🙏 Thank you so much!
My rails application connects to multiple databases with the configs below:
database.yml
``` production: primary_shard: ... secondary_shard: ... queue: ... ```config/initializers/database.rb
```ruby # This is needed due to us using another gem whose model writes to both primary_shard and secondary_shard and its model inherits from ActiveRecord::Base ActiveRecord::Base.instance_eval do connects_to shards: { primary_shard: { writing: :primary_shard, }, secondary_shard: { writing: :secondary_shard, }, } end ```SolidQueuehas been working well, the only time where we cannot enqueue is if we have a block of code like below:I think it is because
SolidQueueis trying to insert the job intosecondary_shardinstead of thequeuedatabase. Is there any way to circumvent this besides rewriting the code to callperform_lateroutside of theconnected_toblock?SomeJob.perform_latermight be called inside a function with theconnected_toblock so it's quite a big refactor for us.