jenkinsci / gitlab-plugin

A Jenkins plugin for interfacing with GitLab
https://plugins.jenkins.io/gitlab-plugin/
GNU General Public License v2.0
1.43k stars 612 forks source link

NPE unless branchFilterType is set - Job DSL issue #870

Open thefirstofthe300 opened 5 years ago

thefirstofthe300 commented 5 years ago

Issue

Context

Logs & Traces

WebHook called with url: /project/<snip>/<snip>
Error while serving http://<snip>/project/<snip>/<snip>
java.lang.NullPointerException
    at com.dabsquared.gitlabjenkins.trigger.filter.BranchFilterFactory.newBranchFilter(BranchFilterFactory.java:15)
    at com.dabsquared.gitlabjenkins.GitLabPushTrigger.initializeBranchFilter(GitLabPushTrigger.java:492)
    at com.dabsquared.gitlabjenkins.GitLabPushTrigger.onPost(GitLabPushTrigger.java:434)
    at com.dabsquared.gitlabjenkins.webhook.build.PushBuildAction$1.performOnPost(PushBuildAction.java:86)
    at com.dabsquared.gitlabjenkins.webhook.build.BuildWebHookAction$TriggerNotifier.run(BuildWebHookAction.java:54)
    at hudson.security.ACL.impersonate(ACL.java:273)
    at com.dabsquared.gitlabjenkins.webhook.build.PushBuildAction.execute(PushBuildAction.java:83)
    at com.dabsquared.gitlabjenkins.webhook.build.BuildWebHookAction.execute(BuildWebHookAction.java:31)
    at com.dabsquared.gitlabjenkins.webhook.GitLabWebHook.getDynamic(GitLabWebHook.java:44)
    at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)
    at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:343)
Caused: java.lang.reflect.InvocationTargetException
    at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:347)
    at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:184)
    at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:372)
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845)
    at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:374)
    at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:715)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:845)
    at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649)
    at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
    at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:860)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
    at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:243)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
    at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
    at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:61)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
    at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:59)
    at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
    at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
    at com.dabsquared.gitlabjenkins.webhook.GitLabWebHook$GitlabWebHookCrumbExclusion.process(GitLabWebHook.java:53)
    at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:73)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
    at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:135)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
    at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
    at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
    at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
    at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
    at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
    at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
    at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
    at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
    at org.eclipse.jetty.server.Server.handle(Server.java:530)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
    at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
    at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
    at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
    at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

Problem description

Describe your problem in a meaningful way:

We are running a setup where Jenkins jobs are set up to be triggered by webhooks on a push to every branch in a repo. We recently began trying to configure the job triggers using the Jenkins DSL plugin to create a job where the triggers are configured in an SCM provided Jenkinsfile instead of configuring the triggers manually. When doing so, we ran into an issue where we would run the pipeline manually once to allow the job to pull in the trigger information.

Example trigger config:

gitlab(triggerOnPush: true, triggerOnMergeRequest: false, triggerOnNoteRequest: true, noteRegex: "jenkins rebuild")

The triggers would then show up as being configured in the web UI. When we pushed to the Github repo, the webhook would show up as having been received by Jenkins in the logs but the job itself had not triggered. After cranking up the debug logging as explained in the README, we stumbled upon the above NPE. Since the NPE appeared to be caused by a null BranchFilterConfig, I changed the triggers config to

gitlab(triggerOnPush: true, triggerOnMergeRequest: false, triggerOnNoteRequest: true, noteRegex: "jenkins rebuild", branchFilterType: 'All')

and the Jenkins jobs were successfully triggered after running the job manually to pick up the new trigger config. I tried tracking down where the config type was not being set correctly but I'm not much good with Java. :\

The Gitlab webhook is configured as follows:

{
    "id": "<ID>", # snipped
    "url": "http://<snip>/project/<FOLDER_NAME>/<JOB_NAME>", # snipped
    "push_events": true,
    "push_events_branch_filter": "",
    "issues_events": false,
    "confidential_issues_events": false,
    "merge_requests_events": false,
    "tag_push_events": false,
    "note_events": false,
    "job_events": false,
    "pipeline_events": false,
    "wiki_events": false,
    "enable_ssl_verification": true,
    "token": ""
}

Let me know if you need any more info.

omehegan commented 5 years ago

We recently began trying to configure the job triggers using the Jenkins DSL plugin

What is the Jenkins DSL plugin? Are you talking about the Job DSL plugin? If so, I don't understand why you are using that in this setup. Also, your before and after example configs are the same.

thefirstofthe300 commented 5 years ago

Yes. I am referring to the Job DSL plugin. The automation setup we have going is as follows: our initial Jenkins setup loads a pipeline discovery automation script using Job DSL. This pipeline discovery script in turn scrapes our source code to discover our Jenkins pipelines and loads each pipeline using a standardized Job DSL template. Each pipeline is customized using the options/environment/triggers/etc set in the Jenkins file. In other words, we have things set up this way to maximize flexibility as much as possible while still making importing pipelines easy.

I also corrected the configs in my original post. The only change I made was to explicitly set the branchFilterType option to All despite branch filters not being used at all.

omehegan commented 5 years ago

This is probably happening because the plugin should set a default value for this parameter in code, but it doesn't bother because the value is normally set by default by the browser on the job config UI. Since Job DSL is generating a config, it is possible for default values to be unset. I don't have a lot of love for Job DSL for this and lots of other related reasons.

I think you could accomplish the same thing by using a properties step in your Jenkinsfiles to define the plugin settings as you want them. See https://github.com/jenkinsci/gitlab-plugin#pipeline-multibranch-jobs-1 This is only strictly necessary for Multibranch jobs, but IIRC it will work fine for regular Pipeline jobs as well, and will supersede whatever is set in the UI. This might be a more robust solution, and might simplify your Job DSL code. Let me know if this is helpful.

You would also benefit from #499, if we can get that done, so you don't need to use this pipeline discovery script.