collectiveidea / delayed_job_mongoid

Mongoid backend for delayed_job
MIT License
170 stars 120 forks source link

Failed query during deserialization should not raise Delayed::DeserializationError #73

Closed johnnyshields closed 6 years ago

johnnyshields commented 6 years ago

In this file:

https://github.com/collectiveidea/delayed_job_mongoid/blob/master/lib/delayed/serialization/mongoid.rb

You can see self.yaml_new raises Delayed::DeserializationError

This is not good when working with replica sets. In rare cases, when triggering a delayed job (e.g. a mailer) on new object creation, the job will be invoked and query a replica set member BEFORE the data has replicated, raising a Mongoid::Errors::DocumentNotFound which is re-raised as a Delayed::DeserializationError

Delayed::DeserializationError is NOT the appropriate error here, since it implies that the handler itself is malformed, and hence no matter how many times we try to deserialize, we will never succeed. DJ handles it by failing the job immediately without retrying. In the case of a failed query however, if we wait ~1 second we WILL succeed.

My proposal is to simply raise the Mongoid::Errors::DocumentNotFound and let the job retry X times.

johnnyshields commented 6 years ago

Will raise this in DelayedJob main repo.