search

timols / jenkins-gitlab-merge-request-builder-plugin

A plugin that allows jenkins to build merge requests in Gitlab
Other
345 stars 127 forks source link

Race condition causing jobs not to be loaded on Jenkins startup #50

Open bjaglin opened 10 years ago

bjaglin commented 10 years ago

Running latest master (and latest stable release), jobs with a gitlab merge request trigger on a https server sometimes disappear after a Jenkins restart, because of this exception (and despite the fact that the ignoreCertificateErrors flag is enabled).

SEVERE: Failed Loading job service-rtb-mr
java.lang.Error: javax.net.ssl.SSLHandshakeException: You can disable certificate checking by setting ignoreCertificateErrors on GitlabHTTPRequestor
        at org.gitlab.api.http.GitlabHTTPRequestor$1.fetch(GitlabHTTPRequestor.java:199)
        at org.gitlab.api.http.GitlabHTTPRequestor$1.hasNext(GitlabHTTPRequestor.java:154)
        at org.gitlab.api.GitlabAPI.getAllProjects(GitlabAPI.java:93)
        at org.jenkinsci.plugins.gitlab.GitlabRepository.getProjectForPath(GitlabRepository.java:98)
        at org.jenkinsci.plugins.gitlab.GitlabRepository.checkState(GitlabRepository.java:39)
        at org.jenkinsci.plugins.gitlab.GitlabRepository.init(GitlabRepository.java:30)
        at org.jenkinsci.plugins.gitlab.GitlabMergeRequestBuilder.build(GitlabMergeRequestBuilder.java:52)
        at org.jenkinsci.plugins.gitlab.GitlabBuildTrigger.start(GitlabBuildTrigger.java:49)
        at org.jenkinsci.plugins.gitlab.GitlabBuildTrigger.start(GitlabBuildTrigger.java:28)
        at hudson.model.AbstractProject.onLoad(AbstractProject.java:331)
        at hudson.model.Project.onLoad(Project.java:90)
        at hudson.model.Items.load(Items.java:276)
        at jenkins.model.Jenkins$18.run(Jenkins.java:2589)
        at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
        at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282)
        at jenkins.model.Jenkins$7.runTask(Jenkins.java:899)
        at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210)
        at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
Caused by: javax.net.ssl.SSLHandshakeException: You can disable certificate checking by setting ignoreCertificateErrors on GitlabHTTPRequestor
        at org.gitlab.api.http.GitlabHTTPRequestor.parse(GitlabHTTPRequestor.java:275)
        at org.gitlab.api.http.GitlabHTTPRequestor.access$200(GitlabHTTPRequestor.java:31)
        at org.gitlab.api.http.GitlabHTTPRequestor$1.fetch(GitlabHTTPRequestor.java:192)
        ... 20 more

Probably a race condition between GitlabBuildTrigger#GitlabBuildTriggerDescriptor#configure() and GitlabBuildTrigger#GitlabBuildTriggerDescriptor#getGitlab() ? As a workaround, setting the initial value of _ignoreCertificateErrors to true fixes the issue...

bjaglin commented 10 years ago

Actually, my workaround doesn't seem to work every time, so there is definitely an annoying race condition. Maybe earlier, during config loading?

kamazee commented 10 years ago

I'm also affected by this issue, but don't have enough experience to debug it. I'd be grateful if anyone can advice anything. Logging (even with messages added here and there around the issue) doesn't help much -- Java Gitlab API seems to get true for ignoring certificate errors, but raises an exception anyway.

However, there's another way out which is probably more secure than ignoring certificate errors: adding a certificate to a storage. This is a fairly simple operation:

  1. Fetch a certificate (Here is a guide)
  2. Find out where your Java installation keeps certificate information (/etc/ssl/certs/java/cacerts for Debian)
  3. # keytool -import -file RETRIEVED_CERT -alias SENSIBLE_ALIAS -keystore CERT_STORAGE
djdefi commented 10 years ago

I attempted to import the cert via keytool, however still recieve the error. Maybe I did something wrong there, will try again when I get a moment.

kamazee commented 10 years ago

@rtrauntvein did it work? It might require restarting Jenkins.

chrode commented 10 years ago

You should check https://github.com/timols/jenkins-gitlab-merge-request-builder-plugin/issues/87#issuecomment-59209892 That solution might be valid too.

dmaslakov commented 9 years ago

Issue still exists even with option 'Ignore SSL Certificate Errors' on. Are there plans to get it fixed?

SEVERE: Failed Loading job merge-build-job
java.lang.Error: javax.net.ssl.SSLHandshakeException: You can disable certificate checking by setting ignoreCertificateErrors on GitlabHTTPRequestor. SSL Error: sun.security.validator.ValidatorException: PKIX path building failed: sun.se
curity.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at org.gitlab.api.http.GitlabHTTPRequestor$1.fetch(GitlabHTTPRequestor.java:218)
        at org.gitlab.api.http.GitlabHTTPRequestor$1.hasNext(GitlabHTTPRequestor.java:174)
        at org.gitlab.api.http.GitlabHTTPRequestor.getAll(GitlabHTTPRequestor.java:143)
        at org.gitlab.api.GitlabAPI.getProjects(GitlabAPI.java:168)
        at org.jenkinsci.plugins.gitlab.GitlabRepository.getProjectForPath(GitlabRepository.java:98)
        at org.jenkinsci.plugins.gitlab.GitlabRepository.checkState(GitlabRepository.java:39)
        at org.jenkinsci.plugins.gitlab.GitlabRepository.init(GitlabRepository.java:30)
        at org.jenkinsci.plugins.gitlab.GitlabMergeRequestBuilder.build(GitlabMergeRequestBuilder.java:52)
        at org.jenkinsci.plugins.gitlab.GitlabBuildTrigger.start(GitlabBuildTrigger.java:44)
        at org.jenkinsci.plugins.gitlab.GitlabBuildTrigger.start(GitlabBuildTrigger.java:27)
        at hudson.model.AbstractProject.onLoad(AbstractProject.java:326)
        at hudson.model.Project.onLoad(Project.java:95)
        at hudson.model.Items.load(Items.java:279)
        at jenkins.model.Jenkins$17.run(Jenkins.java:2673)
        at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
        at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282)
        at jenkins.model.Jenkins$7.runTask(Jenkins.java:903)
        at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210)
        at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: javax.net.ssl.SSLHandshakeException: You can disable certificate checking by setting ignoreCertificateErrors on GitlabHTTPRequestor. SSL Error: sun.security.validator.ValidatorException: PKIX path building failed: sun.security
.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at org.gitlab.api.http.GitlabHTTPRequestor.parse(GitlabHTTPRequestor.java:294)
        at org.gitlab.api.http.GitlabHTTPRequestor.access$200(GitlabHTTPRequestor.java:39)
        at org.gitlab.api.http.GitlabHTTPRequestor$1.fetch(GitlabHTTPRequestor.java:211)
        ... 21 more
chrode commented 9 years ago

Did you try switching to openjdk?

dmaslakov commented 9 years ago

No I did not. If I get it correctly, openjdk will solve the issue with certificate validation due to different cipher length. But it's likely not mine case because after Jenkins was restarted, all tasks were successfully loaded. Looks more like race condition when loading system configuration.

jpappe commented 9 years ago

For whatever it's worth, this hit me after updating my Jenkins server and all plugins to their latest releases:

Several of my jobs failed to load. I restarted Jenkins and a different (smaller) set of jobs failed to load. I then reloaded config from disk and now all my jobs are back.