openzipkin / brave

Java distributed tracing implementation compatible with Zipkin backend services.
Apache License 2.0
2.35k stars 714 forks source link

jms: cleans up artemis code in attempts to deflake test #1394

Closed codefromthecrypt closed 6 months ago

codefromthecrypt commented 6 months ago

This extracts code I started to refactor while trying to get to the bottom of this error on an unrelated PR.

 Error: [ERROR] brave.jakarta.jms.ITJmsTracingMessageConsumer.receive_customSampler -- Time elapsed: 0.028 s <<< ERROR!
[INFO] jakarta.jms.JMSException: AMQ219010: Connection is destroyed
[INFO]  at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:478)
[INFO]  at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:452)
[INFO]  at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.createSessionContext(ActiveMQClientProtocolManager.java:308)
[INFO]  at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager.createSessionContext(ActiveMQClientProtocolManager.java:256)
[INFO]  at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.createSessionChannel(ClientSessionFactoryImpl.java:1523)
[INFO]  at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.createSessionInternal(ClientSessionFactoryImpl.java:837)
[INFO]  at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.createSession(ClientSessionFactoryImpl.java:341)
[INFO]  at org.apache.activemq.artemis.jms.client.ActiveMQConnection.authorize(ActiveMQConnection.java:650)
[INFO]  at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnectionInternal(ActiveMQConnectionFactory.java:925)
[INFO]  at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:290)
[INFO]  at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:285)
[INFO]  at brave.jakarta.jms.ArtemisJmsExtension.newConnection(ArtemisJmsExtension.java:69)
[INFO]  at brave.jakarta.jms.JmsExtension.beforeEach(JmsExtension.java:82)
[INFO]  at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
[INFO]  at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
[INFO] Caused by: ActiveMQNotConnectedException[errorType=NOT_CONNECTED message=AMQ219010: Connection is destroyed]
[INFO]  ... 15 more
[INFO] 

@reta this is on a shared branch, if you can see anything we can do to avoid flakes in activemq, please give a try. Note the failure was in the jarkarta invoker tests, and possibly resource related in CI. If there's a way to make things leaner maybe it will pass. Also, if there'a another MQ we don't need to stick with artemis either. In zipkin and reporter, I moved to docker for reasons like this. However, since this is instrumenting, I'm not sure if we can or not.. I guess we could as we arent' instrumenting the broker... Anyway, if you have some time to give a try, please do as this flake is locking up the project. If we can use docker, yank the ActiveMQExtension from zipkin or reporter and swap it out!

codefromthecrypt commented 6 months ago

also, for some reason, in the other PR only the JDK11 run of the jms-jakarta invoker test was flaking, 21 hasn't so far.

reta commented 6 months ago

also, for some reason, in the other PR only the JDK11 run of the jms-jakarta invoker test was flaking, 21 hasn't so far.

This is very interesting ... shouldn't be JDK dependent ...

reta commented 6 months ago

here are some notes in case helps. If you don't have time, I can swing the axe again tomorrow.

@codefromthecrypt happy to look into that (it may took me a few days), we have fought a few flaky tests in Apache CXF related to Artemis migration

codefromthecrypt commented 6 months ago

@reta agree to merge this. I'm wondering if CI was having a bad day. In any case, we can revisit later as isn't currently a blocker.

codefromthecrypt commented 6 months ago

opened an issue as the flake returned https://github.com/openzipkin/brave/actions/runs/7438289752/job/20236893811?pr=1395

https://github.com/openzipkin/brave/issues/1398