search

jondot / sneakers

A fast background processing framework for Ruby and RabbitMQ
https://github.com/jondot/sneakers
MIT License
2.24k stars 333 forks source link

Not StandardError exception during error reporting #412

Closed sharshenov closed 3 years ago

sharshenov commented 4 years ago

Hello, I've found a bug that makes Sneakers not to report acknowledge state on error.

Wiki suggests to pass context_hash to error notification agents. link

Sneakers.error_reporters << proc { |exception, _worker, context_hash| Honeybadger.notify(exception, context_hash) }

I'm using Sentry raven for error notifications. Sometimes it fails to serialize context_hash to json because Bunny::Channel instance in delivery_info of context_hash causes

SystemStackError: stack level too deep

As result:

  1. RabbitMQ keeps message as Unacked
  2. Such messages number may reach number of prefetched messages for worker
  3. Worker hangs without performing any workload

Sneakers handles only StandardError and its descendants here and can not handle SystemStackError because it is not a StandardError.

Previously I fixed similar bug in another place. And I would be happy to do the same here.

But I think that it would be better to rescue from Exception in both places and re-raise exception in case of Interrupt, NoMemoryError and other exceptions that should not be swallowed.

WDYT?