ScalaConsultants / reactive-rabbit

Reactive Streams driver for AMQP protocol. Powered by RabbitMQ library.
Apache License 2.0
184 stars 40 forks source link

Support callbacks on connection or channel Shutdown and connection Blocking #52

Open michaelzg opened 7 years ago

michaelzg commented 7 years ago

Ref RabbitMQ Java Client 3.6.1: ShutdownListener BlockedListener

mkiedys commented 7 years ago

Can you explain why do you need this feature?

michaelzg commented 7 years ago

@mkiedys Sure,

This is useful for monitoring and reacting when the connection to rabbit is down. (Logging, counting, prompt a reconnect). A workaround without this functionality, was to manually do a source.tick to generate a "heartbeat" of the RabbitConnection. That's inefficient and the detection is only as frequent as the tick interval.

Similar can be said for a blocked connection, it is very useful to me for monitoring.

@djamelz mentioned on Gitter advocating for this as well, maybe he can provide context for his motivation as well.

djamelz commented 7 years ago

Yes, i'm currently use it in production (via fork) to ensure the "based on callback" reconnection to rabbit. Combined with a retry mechanism it works very well !

mkiedys commented 7 years ago

Have you looked at supervision strategies? http://doc.akka.io/docs/akka/2.4.12/scala/stream/stream-error.html

michaelzg commented 7 years ago

Yes, the supervision strategies were considered but found to be not optimal. Here are the two options I see in using supervision strategies and both do not come close to what the attached listeners can accomplish (but please let me know see something I don't).

Also worth mentioning a blocked connection (and its blocked listener) is useful to know and not something that stream supervision can portray explicitly.

  1. Separate Heartbeat stream for RabbitMQ connection. as mentioned previously via a source.tick, is inefficient and only as quick as the tick interval. Upon a down'ed RabbitMQ connection, this heartbeat tick stream will error out and the supervision restart logic will be in the handler--that same logic would easily slide right in the shutdown listener.

  2. Supervise the Publish/Subscribe stream and attempt to re-establish connection. The listeners will do a much better job than this option as well. I see two scenarios:

[A] when we have a fair number of pub/sub streams going into a rabbit connection (imagine having to restart thousands, multiple times until connection re-established so everyone is waiting and backing up). All those clients will have to raise a flag screaming at, say, an Actor to re-establish the connection. I'd rather see that Actor be able to monitor the RabbitMQ connection on it's own via the point above--but even better would be the listener.

[B] when the publish frequency is long (e.g. several minutes) and a down'd RabbitMQ connection will only be detected and acted upon when a publish fails. We can be more proactive with listeners to detect the trouble sooner and recover faster.

arosien commented 7 years ago

I have a consulting client that uses this patch and having this merged into an official release would be fantastic! Anything I can do to help make that happen?

michaelzg commented 7 years ago

I can help resolve the branch conflicts that arose over the months.