search

jenkinsci / jms-messaging-plugin

https://plugins.jenkins.io/jms-messaging
12 stars 38 forks source link

Add simple client for RabbitMQ #154

Closed Zlopez closed 4 years ago

Zlopez commented 4 years ago

I'm opening this PR as continuation of #147

This client is based on RabbitMQ Java Client.

Supports SSL and Username/Password authentication and could be configured to work with any RabbitMQ server.

Connection to each server is created only once and separate channel is created for each consumer and publisher.

When publishing generates message id using java.util.UUID and send this id as property of the message.

Consuming is done using by using Push API (subscription) even when listening for messages for a limited time.

Signed-off-by: Michal Konečný mkonecny@redhat.com

Zlopez commented 4 years ago

@scoheb I'm still hitting the issue I mentioned in the previous PR, so I'm unable to run tests locally. Any advice?

scoheb commented 4 years ago

@Zlopez Sorry...I forget what the original issue was. Can you describe again here please?

scoheb commented 4 years ago

did you examine the failures here though: https://ci.jenkins.io/blue/organizations/jenkins/Plugins%2Fjms-messaging-plugin/detail/PR-154/1/tests ??

Zlopez commented 4 years ago

No I didn't, I'm still trying to run them on my local machine. But I'm unable to run them because of the error bellow. I'm trying to run this command mvn -B test -DElasticTime.factor=2 -Djenkins.version=2.176.1 -DforkCount=1 -Dtest=FedMsgMessagingPluginIntegrationTest#testSimpleCIEventTriggerWithCheck

Building Docker image `docker build -t jenkins/fedmsg-relay:e048987d8a07 /tmp/Dockerfile5088748677418623093dir`: logfile is at target/diagnostics/testSimpleCIEventTriggerWithCheck(com.redhat.jenkins.plugins.ci.integration.FedMsgMessagingPluginIntegrationTest)/docker-FedmsgRelayContainer.build.log
Launching Docker container `docker run -d -p 127.0.0.1::4001 -p 127.0.0.1::2003 -p 127.0.0.1::22 jenkins/fedmsg-relay:e048987d8a07`: logfile will be at target/diagnostics/testSimpleCIEventTriggerWithCheck(com.redhat.jenkins.plugins.ci.integration.FedMsgMessagingPluginIntegrationTest)/docker-FedmsgRelayContainer.run.log
Lis 01, 2019 4:25:03 ODP. org.jenkinsci.test.acceptance.update_center.MockUpdateCenter lambda$serverExceptionHandler$5
WARNING: Exception thrown while serving request
java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:210)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
    at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
    at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
    at org.apache.http.impl.io.DefaultHttpRequestParser.parseHead(DefaultHttpRequestParser.java:129)
    at org.apache.http.impl.io.DefaultHttpRequestParser.parseHead(DefaultHttpRequestParser.java:53)
    at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
    at org.apache.http.impl.DefaultBHttpServerConnection.receiveRequestHeader(DefaultBHttpServerConnection.java:129)
    at org.apache.http.protocol.HttpService.handleRequest(HttpService.java:308)
    at org.apache.http.impl.bootstrap.Worker.run(Worker.java:66)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

127.0.0.1:36807
127.0.0.1:41561
master53872|Nov 01, 2019 4:25:38 PM com.redhat.jenkins.plugins.ci.CIBuildTrigger stopTriggerThreads
master53872|INFO: Removed thread lock for 'representative_glow'.
master53872|Nov 01, 2019 4:25:38 PM com.redhat.jenkins.plugins.ci.CIBuildTrigger startTriggerThreads
master53872|INFO: Starting thread (183) for 'representative_glow'.
master53872|Nov 01, 2019 4:25:38 PM com.redhat.jenkins.plugins.ci.ProjectChangeListener onUpdated
master53872|INFO: Getting trigger threads.
master53872|Nov 01, 2019 4:25:39 PM com.redhat.jenkins.plugins.ci.messaging.FedMsgMessagingWorker subscribe
master53872|INFO: Subscribing job 'representative_glow' to org.fedoraproject topic.
master53872|Nov 01, 2019 4:25:39 PM com.redhat.jenkins.plugins.ci.messaging.FedMsgMessagingWorker subscribe
master53872|INFO: Successfully subscribed job 'representative_glow' to topic 'org.fedoraproject'.
master53872|Nov 01, 2019 4:25:53 PM com.redhat.jenkins.plugins.ci.ProjectChangeListener onUpdated
master53872|INFO: No CIBuildTrigger found, forcing thread stop.
master53872|Nov 01, 2019 4:25:53 PM com.redhat.jenkins.plugins.ci.CIBuildTrigger stopTriggerThreads
master53872|INFO: Removed thread lock for 'trivial_liquid'.
master53872|Nov 01, 2019 4:25:56 PM hudson.model.Run execute
master53872|INFO: trivial_liquid #1 main build action completed: SUCCESS
master53872|Nov 01, 2019 4:25:56 PM com.redhat.utils.MessageUtils sendMessage
master53872|INFO: Sending message for job 'trivial_liquid'.
master53872|Nov 01, 2019 4:26:03 PM com.redhat.jenkins.plugins.ci.messaging.JMSMessagingWorker trigger
master53872|INFO: Scheduling job 'representative_glow' based on message:
master53872|{"content":"catch me"}
master53872|Nov 01, 2019 4:26:03 PM hudson.model.Run execute
master53872|INFO: representative_glow #1 main build action completed: SUCCESS
1572621965303   Marionette  INFO    Stopped listening on port 46111

###!!! [Parent][RunMessage] Error: Channel closing: too late to send/recv, messages will be lost

But if you don't know, what is causing this, I will use the CI as authority on this and try to fix the issue directly in this PR.

scoheb commented 4 years ago

But are you sure it failed? I do not see the maven test result summary line. It is normal to see some exceptions in the log messages. I do also see 2 successful builds in the output and that is a good indication that one job triggered the other.

Zlopez commented 4 years ago

@scoheb It looks like you are correct. My bad, I saw the error and thought that the test failed. I will now work on fixing the RabbitMQ tests.

Zlopez commented 4 years ago

Looking at the issue in Jenkins, I'm not exactly sure what is wrong. I have com/redhat/jenkins/plugins/ci/integration/docker/fixtures/JBossRabbitMQContainer in project, so I don't understand why it complains that the directory doesn't exists.

scoheb commented 4 years ago

you are missing the associated Dockerfile for the container...

Zlopez commented 4 years ago

I found only one Dockerfile in the whole project src/test/resources/ath-container/Dockerfile. I don't understand, how I should associate Dockerfile to com/redhat/jenkins/plugins/ci/integration/docker/fixtures/JBossRabbitMQContainer

scoheb commented 4 years ago

In master, I see:

shebert@localhost~/work/jenkins/plugins/jms-messaging-plugin/src:master$ find . | grep Dockerfile
./main/resources/com/redhat/jenkins/plugins/ci/integration/docker/fixtures/JBossAMQContainer/Dockerfile
./main/resources/com/redhat/jenkins/plugins/ci/integration/docker/fixtures/FedmsgRelayContainer/Dockerfile
./test/resources/ath-container/Dockerfile
Zlopez commented 4 years ago

Oh my bad, I looked in /src/test/ folder only.

Zlopez commented 4 years ago

I added the Dockerfile and the test is now running, but I'm having a weird issue. The plugin is installed, but it doesn't have RabbitMQ configure option. I think it doesn't use local version, what should I do to force it?

scoheb commented 4 years ago

you need to run:

mvn clean install -DskipTests

first since the plugin needs to be deployed to the local maven repo for ATH to pick it up.

Zlopez commented 4 years ago

This must be done after every change in the code?

scoheb commented 4 years ago

only when you change the plugin code. not the tests.

Zlopez commented 4 years ago

Ok, I did mvn clean install -DskipTests and the option is still missing on configure page.

scoheb commented 4 years ago

can you pastebin both mvn command outputs?

Zlopez commented 4 years ago

mvn clean install -DskipTests: https://paste.centos.org/view/cb3be7e0 mvn -B test -DElasticTime.factor=2 -Djenkins.version=2.176.1 -DforkCount=1 -Dtest=RabbitMQMessagingPluginIntegrationTest#testSimpleCIEventTrigger: https://paste.centos.org/view/8262b751

scoheb commented 4 years ago

it is picking up a later release 1.1.11 in the upstream update center....while your local snapshot is 1.10...

you can update your pom to be 1.12.0-snapshot or rebase to master...

Zlopez commented 4 years ago

Thanks, I didn't noticed that.

scoheb commented 4 years ago

@Zlopez making good progress?

Zlopez commented 4 years ago

@scoheb I was able to succesfully run most of the tests. Now I'm trying to figure out four pipeline tests that are failing with NullPointerException.

When I will be done, I will squash all the commits and let you know.

Zlopez commented 4 years ago

It is possible to disable CI for a while, so I don't trigger it each time I'm spam a new commit?

scoheb commented 4 years ago

close the MR ... or create a new local branch for yourself.

Zlopez commented 4 years ago

All tests should pass now. I squashed all commits and will wait till the CI will be done with it.

Zlopez commented 4 years ago

I found one issue with queue override, so I fixed it and add a new test for this.

Zlopez commented 4 years ago

How long till this one will be merged? I already have some small change that needs this PR in master.

scoheb commented 4 years ago

Available in version 1.1.12