search

kuleuven / jenkins-mattermost-plugin

Jenkins plugin for Mattermost
MIT License
24 stars 46 forks source link

Posting in Mattermost not working #27

Closed hathagat closed 5 years ago

hathagat commented 6 years ago

Hi,

in my environment Jenkins and Mattermost (GitLab Omnibus) are running on two different servers (CentOS, OpenJDK 8, no Docker) with self signed certificates. I added the certificates to the JKS and the appropriate Mattermost, Jenkins and OS folders on both servers.

Jenkins Settings:

Endpoint: https://mymattermost.local:8066/hooks/123xyz Channel: jenkins-testing Icon: empty Build Server URL: https://myjenkins.local

When I run the connection test in the Jenkins log shows the following:

May 23, 2018 1:13:39 PM INFO jenkins.plugins.mattermost.StandardMattermostService publish
Posting: to jenkins-testing@https://mymattermost.local:8066/hooks/123xyz: Mattermost/Jenkins plugin: you're all set! (parameters: endpoint='https://mymattermost.local:8066/hooks/123xyz', room='jenkins-testing', icon='', buildServerUrl='https://myjenkins.local/') (good)
May 23, 2018 1:13:39 PM INFO org.apache.commons.httpclient.HttpMethodBase readResponseBody
Response content length is not known
May 23, 2018 1:13:39 PM INFO org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
I/O exception (javax.net.ssl.SSLPeerUnverifiedException) caught when processing request: peer not authenticated
May 23, 2018 1:13:39 PM INFO org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
Retrying request
May 23, 2018 1:13:39 PM INFO org.apache.commons.httpclient.HttpMethodBase readResponseBody
Response content length is not known
May 23, 2018 1:13:39 PM INFO org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
I/O exception (javax.net.ssl.SSLPeerUnverifiedException) caught when processing request: peer not authenticated
May 23, 2018 1:13:39 PM INFO org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
Retrying request
May 23, 2018 1:13:39 PM INFO org.apache.commons.httpclient.HttpMethodBase readResponseBody
Response content length is not known
May 23, 2018 1:13:39 PM INFO org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
I/O exception (javax.net.ssl.SSLPeerUnverifiedException) caught when processing request: peer not authenticated
May 23, 2018 1:13:39 PM INFO org.apache.commons.httpclient.HttpMethodDirector executeWithRetry
Retrying request
May 23, 2018 1:13:39 PM INFO org.apache.commons.httpclient.HttpMethodBase readResponseBody
Response content length is not known
May 23, 2018 1:13:39 PM WARNING jenkins.plugins.mattermost.StandardMattermostService publish
Error posting to Mattermost
javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
    at sun.security.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:431)
    at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.verifyHostName(SSLProtocolSocketFactory.java:257)
    at org.apache.commons.httpclient.protocol.SSLProtocolSocketFactory.createSocket(SSLProtocolSocketFactory.java:200)
    at org.apache.commons.httpclient.HttpConnection.tunnelCreated(HttpConnection.java:793)
    at org.apache.commons.httpclient.HttpMethodDirector.executeConnect(HttpMethodDirector.java:521)
    at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
    at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:178)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:404)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:330)
    at jenkins.plugins.mattermost.StandardMattermostService.publish(StandardMattermostService.java:99)
    at jenkins.plugins.mattermost.StandardMattermostService.publish(StandardMattermostService.java:41)
    at jenkins.plugins.mattermost.MattermostNotifier$DescriptorImpl.doTestConnection(MattermostNotifier.java:452)
    at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)
    at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:343)
    at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:184)
    at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:117)
    at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:129)
    at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845)
    at org.kohsuke.stapler.MetaClass$5.doDispatch(MetaClass.java:248)
    at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649)
    at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:841)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
    at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
    at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:99)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
    at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:198)
    at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
    at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:199)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.jenkinsci.plugins.cas.spring.security.CasSingleSignOutFilter.doFilter(CasSingleSignOutFilter.java:39)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
    at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
    at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
    at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
    at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
    at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
    at org.eclipse.jetty.server.Server.handle(Server.java:564)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:317)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
    at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
    at org.eclipse.jetty.util.thread.Invocable.invokePreferred(Invocable.java:128)
    at org.eclipse.jetty.util.thread.Invocable$InvocableExecutor.invoke(Invocable.java:222)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:294)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:199)
    at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
May 23, 2018 1:13:39 PM INFO jenkins.plugins.mattermost.StandardMattermostService publish
Posting succeeded

The gitlab_mattermost_access log shows nothing in this case.

Testing from Jenkins server using curl works as expected: curl -i -X POST -d 'payload={"text": "Hello\nText"}' https://mymattermost.local:8066/hooks/123xyz Here the gitlab_mattermost_access log shows "POST /hooks/123xyz HTTP/1.1" 200 2 "" "curl/7.58.0"

Any idea why the connection fails? Thanks in advance!

jovandeginste commented 6 years ago

did you look at #12 and #16 ? Both are related to SSL issues. The last one may very well be the solution - or not, but I'd appreciate you verify this first :-)

hathagat commented 6 years ago

Speaking of #12 I use OpenJDK 1.8 so there souldn't be a problem. Additionally I verified that TLSv1 is activated. The certificates (whole chain including root CA) are already imported to the Java and the Jenkins Keystores and copied to the following folders /etc/pki/ca-trust/source/anchors/ /etc/pki/tls/certs/ $JENKINS_HOME/.keystore/ The webservers on both servers use the whole chain like mentioned in #16.

However I don't understand why the logs on the Mattermost server don't show any connection attempts.

jovandeginste commented 6 years ago

They may not show any attempts if the SSL negotiation does not work...

Since this is a recurring issue, I looked for a more generic solution; maybe this is something to try? https://support.cloudbees.com/hc/en-us/articles/217309497-Test-a-SSL-connection-from-Jenkins

hathagat commented 6 years ago

I just wonder because every other connection between the two servers works as ecxpected.

Thanks for the link! I set up the pipeline and did some testing. There seem to be handshake problems with TLSv1.2. I will investigate further...

jovandeginste commented 6 years ago

Did you find anything that still points at this plugin?

jovandeginste commented 5 years ago

I will close this for inactivity