gaffneyc / sunspot-queue

Background search indexing using existing worker systems
MIT License
35 stars 37 forks source link

Adding support for performing the indexing in real time in case an error occurs when enqueueing the job. #21

Closed guilleva closed 9 years ago

guilleva commented 9 years ago

In my case, sometimes the enqueue fails because or redis timeouts (Redis::TimeoutError: Connection timed out) and because indexing jobs are critical this gives us a way to not lose them and to index the element is real time (which in most of the cases is better than just not indexing them).

gaffneyc commented 9 years ago

Thank you for the pull request and sorry for the long wait on a response. While I like the idea in general it reaches a bit too much into each implementation.

What I would suggest instead is a fallback backend which would be given the appropriate backend.

e.g.

backend = Sunspot::Queue::Fallback::Backend.new(Sunspot::Queue::Sidekiq::Backend.new)
Sunspot.session = Sunspot::Queue::SessionProxy.new(Sunspot.session, backend)

The fallback backend should be able to hold the logic for retrying the enqueue or doing an immediate index should there be a network hiccup.