search

jenkinsci / jms-messaging-plugin

https://plugins.jenkins.io/jms-messaging
12 stars 37 forks source link

Failover broker URL does not failover subscribed jobs when one of the brokers is no longer available #237

Open pholica opened 2 years ago

pholica commented 2 years ago

Jenkins and plugins versions report

Environment Jenkins: 2.303.3 OS: Linux - redacted --- (redacted other plugins due to security concerns) jms-messaging:1.1.26

What Operating System are you using (both controller, and any agents involved in the problem)?

RHEL-7

Reproduction steps

  1. Configure brokers using failover url like: failover:(ssl://01.example.com:61616,ssl://02.example.com:61616)
  2. Create a job that is triggered by CI message
  3. Check which broker was used for subscription
  4. Make the broker inaccessible (e.g. using iptables)

Expected Results

Job is re-subscribed using the other broker configured in the failover URL

Actual Results

The plugin tries resubscribing again and again using the broker that was used for the initial subscribe: Following messages are available in log:

2022-05-29 03:56:55.543+0000 [id=312088]        WARNING o.a.a.t.f.FailoverTransport#doReconnect: Failed to connect to [ssl://01.example.com:61616] after: 4780 attempt(s) with Connection refused (Connection refused), continuing to retry.

Anything else?

I'm not sure if it's somehow related, but the first issue we hit with the broker before it went fully offline (refusing connections) was:

2022-05-23 18:45:02.019+0000 [id=3095047]       WARNING o.a.a.t.f.FailoverTransport#handleTransportFailure: Transport (ssl://01.example.com:61616) failed, attempting to automatically reconnect
java.io.EOFException
        at java.io.DataInputStream.readInt(DataInputStream.java:392)
        at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:268)
        at org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:240)
        at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:232)
        at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
        at java.lang.Thread.run(Thread.java:748)