search

SpinGo / op-rabbit

The Opinionated RabbitMQ Library for Scala and Akka
Other
232 stars 73 forks source link

Restore a faulty subscription #138

Open 1van3 opened 6 years ago

1van3 commented 6 years ago

Hello, a few days ago a colleague of mine has encountered a strange behavior - sometime channel subscription "hangs".

I have tried to investigate an issue and here is my observations:

  1. Subscription "hangs" because default channel Rpc timeout is 10 minutes (see com.rabbitmq.client.ConnectionFactory.DEFAULT_CHANNEL_RPC_TIMEOUT) and there was no ACK on DeclareQueue method from Rabbit side sometimes.
  2. After i have changed DEFAULT_CHANNEL_RPC_TIMEOUT to 1 second (btw there is no way to do it in current version of op-rabbit (see com.spingo.op_rabbit.ConnectionParams)) i was expecting that faulty subscription will recover automatically, but got unhandled TimeoutException (probably for the same reason - because Rabbit hasn't ACKed Close message) java.util.concurrent.TimeoutException at com.rabbitmq.utility.BlockingCell.get(BlockingCell.java:77) at com.rabbitmq.utility.BlockingCell.uninterruptibleGet(BlockingCell.java:120) at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:36) at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:494) at com.rabbitmq.client.impl.ChannelN.close(ChannelN.java:604) at com.rabbitmq.client.impl.ChannelN.close(ChannelN.java:529) at com.rabbitmq.client.impl.ChannelN.close(ChannelN.java:522) at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.newmotion.akka.rabbitmq.RabbitMqActor$class.closeIfOpen(RabbitMqActor.scala:27) at com.newmotion.akka.rabbitmq.ChannelActor.closeIfOpen(ChannelActor.scala:29)

Guys, can you suggest, what should i do in such case? Should i try to kill subscription/rabbit control or restore should be performed on op-rabbit side? If it should be done on my side please suggest how to correctly stop subscription.

timcharper commented 6 years ago

Did you try calling subscriptionRef.abort?

https://github.com/SpinGo/op-rabbit/blob/master/core/src/main/scala/com/spingo/op_rabbit/SubscriptionRef.scala#L35