JENKINS-48885: JenkinsRule always returns null in getPluginManager().getPlugin() on new core versions

[develmac]

Sadly somehow workflow-aggregator plugin can not be found, even if it is on the plugin dependencies list.

pluginDependencies(Action {
        dependency("org.jenkins-ci.plugins.workflow", "workflow-aggregator", "2.5")
        dependency("org.jenkins-ci.plugins", "job-dsl", "1.69")
        dependency("org.6wind.jenkins", "lockable-resources", "2.2")
        dependency("org.jenkinsci.plugins", "pipeline-model-api", "1.2.5")
        dependency("org.jenkinsci.plugins", "pipeline-model-declarative-agent", "1.1.1")
        dependency("org.jenkinsci.plugins", "pipeline-model-definition", "1.2.5")
        dependency("org.jenkinsci.plugins", "pipeline-model-extensions", "1.2.5")
    })

Spock Test:

 given:
            FreeStyleProject freeStyleProject = rule.createFreeStyleProject('project')
            def scripts = new ExecuteDslScripts()
scripts.scriptText = '''
pipelineJob("testPipeline") {}
""".stripIndent())
freeStyleProject.getBuildersList().add(scripts)

 when:
            QueueTaskFuture<WorkflowRun> futureRun = freeStyleProject.scheduleBuild2(0)
 then:
            // JenkinsRule has different assertion capabilities
            WorkflowRun run = rule.assertBuildStatusSuccess(futureRun)
            rule.assertLogContains('''test'''.stripIndent(), run)

Error:

ERROR: (script, line 2) plugin 'workflow-aggregator' needs to be installed

[develmac]

What I figured out is that the jobDSL plugin gets a pluginmanager that has zero plugins (TestPluginManager), I am guessing this is some config issue?

[mkobit]

@maconic I'll hopefully be able to look into this at some point this weekend. It is possible that is in how dependency management for Jenkins plugins is managed now. Is the Spock test you added complete? Maybe I messed it up with my edit, but it looks like it is missing ''' close quotes for scripts.scriptText

[develmac]

I am afraid my knowledge of the jenkins internals is too limited to resolve this :(

Sry, I will post the Spock Spec again!

 def "should create pipelineJob"() {
        given:
            FreeStyleProject freeStyleProject = rule.createFreeStyleProject('project')
            def scripts = new ExecuteDslScripts()

            scripts.scriptText = '''
pipelineJob("testPipeline") {

// because stash notifier will not work
triggers {
    scm('')
}

logRotator {
    numToKeep(15)
    artifactNumToKeep(1)
}

}
    '''.stripIndent()
            freeStyleProject.getBuildersList().add(scripts)

        when:
            QueueTaskFuture<FreeStyleBuild> futureRun = freeStyleProject.scheduleBuild2(0)
        then:
            // JenkinsRule has different assertion capabilities
            def run = rule.assertBuildStatusSuccess(futureRun)
            rule.assertLogContains('''test'''.stripIndent(), run)
    }

[develmac]

It would be great if you can give me a hint over the weekend! I will check here, in case there are some questions.

[develmac]

An even simpler version with Spock would be:

    def "should create pipelineJob"() {
        given:
            WorkflowJob workflowJob = rule.createProject(WorkflowJob, 'project1')
            FreeStyleProject freeStyleProject = rule.createFreeStyleProject('project')

        new DslScriptLoader(new JenkinsJobManagement(System.out, [:], new File('.'))).runScript('''
               pipelineJob('myJob') {

        }
    '''.stripIndent())

        when:
            QueueTaskFuture<FreeStyleBuild> futureRun = freeStyleProject.scheduleBuild2(0)
        then:
            def run = rule.assertBuildStatusSuccess(futureRun)
            rule.assertLogContains('''test'''.stripIndent(), run)
    }

[mkobit]

Can you post full stacktrace from Gradle or from tests that is causing it?

[develmac]

There isn't too much of a stracktrace, but I will try. Maybe I can provide you with an example project if that would be helpful?

[mkobit]

A reproducible project and steps will be very helpful and appreciated!

[develmac]

Sorry for taking me a while, I hope it is going to be helpful.

Just run

gradle integrationTest

(Java needs to be installed, nothing more - I tested it with JDK8)

jobds-problem.zip

[mkobit]

This is somewhat strange, and I don't know why this is happening:

javaposse.jobdsl.dsl.DslScriptException: (script, line 2) plugin 'workflow-aggregator' needs to be installed
    at javaposse.jobdsl.plugin.JenkinsJobManagement.failOrMarkBuildAsUnstable(JenkinsJobManagement.java:394)
    at javaposse.jobdsl.plugin.JenkinsJobManagement.requirePlugin(JenkinsJobManagement.java:281)
    at script.run(script:2)
    at javaposse.jobdsl.dsl.AbstractDslScriptLoader.runScript(AbstractDslScriptLoader.groovy:132)
    at javaposse.jobdsl.dsl.AbstractDslScriptLoader.runScriptEngine(AbstractDslScriptLoader.groovy:106)
    at javaposse.jobdsl.dsl.AbstractDslScriptLoader.runScripts_closure1(AbstractDslScriptLoader.groovy:59)
    at groovy.lang.Closure.call(Closure.java:414)
    at groovy.lang.Closure.call(Closure.java:430)
    at javaposse.jobdsl.dsl.AbstractDslScriptLoader.runScripts(AbstractDslScriptLoader.groovy:46)
    at javaposse.jobdsl.dsl.AbstractDslScriptLoader.runScript(AbstractDslScriptLoader.groovy:85)
    at seeding.JenkinsGlobaLibSpec.should create pipelineJob(JenkinsGlobaLibSpec.groovy:37)

            println("Jenkins Plugins: ${rule.jenkins.pluginManager.plugins.size()}")
            println("Rule Plugins: ${rule.pluginManager.plugins.size()}")

both show 0 which is surprising - I'm going to have to spend some more time to actually investigate why this is happening.

[develmac]

Well for the job dsl plugin to work, you have to create a freestyle job and most examples I have seen work with a workflow job, maybe this is the root issue?

[mkobit]

Test can be reduced down to

  def "should create pipelineJob"() {
    when:
    new DslScriptLoader(new JenkinsJobManagement(System.out, [:], new File('.'))).runScript(
        '''
          //pipelineJob('pipelineJobFails')
          freeStyleJob('freeStyleJobSucceeds')
        '''.stripIndent()
    )

    then:
    rule.jenkins.jobNames.size() == 1
  }

If pipelineJob is uncommented, this fails with the stack trace from above.

So, it doesn't seem like the plugin detection is working. The Job DSL Plugin does some preventative actions to check if plugins are installed and what not before creating job (as seen at https://github.com/jenkinsci/job-dsl-plugin/blob/bfd0dee59f365d6d5223632f6dc210b30f2bb958/job-dsl-core/src/main/groovy/javaposse/jobdsl/dsl/DslFactory.groovy#L120-L121). freeStyleJob('freeStyleJobSucceeds') succeeds while pipelineJob('pipelineJobFails')

Side note, the Job DSL Plugin doesn't require a freestyle project to run, I think that is just how most people make use of it.

For example, you can execute the Job DSL using pipeline (WorkflowJob) by using the jobDsl step

node {
  jobDsl(
    scriptText: '''
      freeStyleJob('jobName')
    '''
  )
}

Right now, I have the dependency management setup so that the JenkinsRule discovers the Jenkins plugins from the classpath. My guess is pointing towards one of a few things:

• JenkinsRule doesn't properly signal which plugins it has loaded this way (Jenkins.getInstance().pluginManager.plugins.size() reports 0)
• Job DSL Plugin uses wrong methods for plugin lookup (I don't think this is it, I see Jenkins.getInstance().getPlugin(pluginShortName) which looks like the right way to me)
• This Gradle plugin is not setting up the dependencies correctly for the JenkinsRule test harness to pick them up

I'm leaning towards the JenkinsRule not properly signaling the plugins it has loaded, but I'll have to look a bit deeper into it.

Side note, there is an example at https://github.com/sheehan/job-dsl-gradle-example for just testing out Job DSL scripts, but you should be able to something similar with this Gradle plugin. That example copies the .jpi and .hpi artifacts directly rather than using classpath, so JenkinsRule might be behaving differently.

[develmac]

Hi!

You are absolutely right, I could get this running by using a workflow script!

I would prefer the approach without copying the plugins! :)

[mkobit]

I believe this is an upstream bug in Jenkins - https://issues.jenkins-ci.org/browse/JENKINS-48885

[develmac]

Looks like it!

[develmac]

Seems like there is not too much of a progress on the Jenkins issue :( Mabye we can get the plugin provider to use a different API?

[develmac]

I guess there is no example on how to use the "copy plugins" approach from Kotlin? TBH I struggle a bit to convert the zero typed Groovy code to fully typed Kotlin :(

[mkobit]

I think the "copy plugins" approach will still have the same issue.

You could try using an older Jenkins core / Jenkins Test Harness version and see if that helps, but I think this needs to be fixed in Jenkins proper.

[develmac]

Yeah I did a downgrade and the older version of the JobDsl plugin treats this as a warning.

[jordanjennings]

Any thoughts on this one two years later? :) I've been avoiding updating for a while because I was hitting this issue and was having trouble figuring out what was going on. In my case I've hit this because the durable task plugin's BourneShellScript.java started internally calling jenkins.getPluginManager().getPlugin("durable-task") and that breaks any integration test that calls the sh step... which is almost all of mine.

When I debug I can see that the plugin manager thinks there's nothing loaded:

I can't figure out any workaround apart from downgrading workflowDurableTaskStepPluginVersion to 2.34

[mkobit]

I think this is basically still relying on upstream fixes in the Jenkins Test Harness (at least from my understanding):

• https://issues.jenkins-ci.org/browse/JENKINS-48885
• https://github.com/jenkinsci/jenkins-test-harness/pull/92

I haven't released a new version in a while with default version updates, but it still doesn't look fixed.

[basil]

I, too, am experiencing this issue with recent versions of the JUnit plugin. I can reproduce this issue with Jenkins Pipeline Shared Library Gradle Plugin 0.10.1 and Jenkins Test Harness 2.64. I cannot reproduce this problem with a Maven-based test, also using Jenkins Test Harness 2.64. So Jenkins Test Harness 2.64 does support calling PluginManager#getPlugin, at least for Maven-based tests. The problem seems specific to Gradle-based tests.

I stepped through the working Maven-based version and compared it to the broken Gradle-based version. The difference seems to be in lines 125-156 of UnitTestSupportingPluginManager, which "pick[s] up test dependency *.jpi [files] that are placed by maven-hpi-plugin['s] TestDependencyMojo and cop[ies] them into $JENKINS_HOME/plugins." The index file it uses to do this is not present for Gradle-based tests, so logic code does not run. This means the plugins do not get copied into $JENKINS_HOME/plugins and therefore do not get registered later in PluginManager when Jenkins is starting up.

The code that creates this index in maven-hpi-plugin is in TestDependencyMojo. Similar code also exists in gradle-hpi-plugin in TestDependenciesTask. But when running tests with Jenkins Pipeline Shared Library Gradle Plugin 0.10.1, I did not see it invoking this task, and I also did not see a test-dependencies directory getting created in build/. This seems to be at the heart of the problem.

To summarize, I am not sure the problem is upstream in the Jenkins test harness. I think the problem is that we are not invoking the logic in TestDependenciesTask when Jenkins Pipeline Shared Library Gradle Plugin is used. I do not know for sure whether the issue is in gradle-hpi-plugin, jenkins-test-harness, this repository, or some interaction between them. Perhaps the maintainer can investigate further using the information I provided above.

[mkobit]

Thanks for looking into it @basil . I haven't spent really any time on this repository in a while since I haven't used Jenkins Pipelines in some time.

Maybe something similar could be done as TestDependencyMojo. Before, it seemed it everything was being scanned from the classpath by Jenkins for plugins and what not, so nothing extra seemingly needed to be done. However, maybe there are a few extra steps that the plugin should take care of.

[AnEmortalKid]

@mkobit @basil I was able to get around this, I think it has to do with the creation of the index file.

We have something like this in our build.gradle:

task resolveTestPlugins(type: Copy) {
    from configurations.testPlugins
    into new File(sourceSets.test.output.resourcesDir, 'test-dependencies')
    include '*.hpi'
    include '*.jpi'

    doLast {
        def baseNames = source.collect { it.name[0..it.name.lastIndexOf('.')-1] }
        new File(destinationDir, 'index').setText(baseNames.join('\n'), 'UTF-8')
    }
}

test {
    dependsOn tasks.resolveTestPlugins
    inputs.files sourceSets.jobs.groovy.srcDirs

    // set build directory for Jenkins test harness, JENKINS-26331
    systemProperty 'buildDirectory', project.buildDir.absolutePath
}

I think we copied this from here a while ago. That drops index contents with the VERSIONS attached:

script-security-1.74

I noticed that the plugin manager was trying to read things through the short name, so script-security. I checked the result of loadBundledPlugins from the TestPluginManager which ends up putting things under the /plugins directory and it had something like this:

trilead-api
trilead-api-1.0.8
trilead-api-1.0.8.jpi
trilead-api.jpi

In my case when it loads trilead-api, it was loading the non versioned one which has an outdated version.

In the interim, I created my own rule/plugin manager just to debug through this more easily (that's how i noticed what archive the Plugin reference was for)

class OverridenRule extends JenkinsRule {

    public static final PluginManager INSTANCE;

    public MyRule() {
        // visible
    }

    static {
        try {
            INSTANCE = new OverridenManager();
        } catch (IOException e) {
            throw new Error(e);
        }
    }

    @Override
    public PluginManager getPluginManager() {
        return INSTANCE;
    }

    static class OverridenManager extends TestPluginManager {

        public OverridenManager() throws IOException {
        }

        @Override
        protected Collection<String> loadBundledPlugins() throws Exception {
          // Overridden method that is going to rename trilead-api-1.0.8.hpi to trilead-api.jpi
            def names = []
            def directory = getClass().getClassLoader().getResource("test-dependencies/")
            def dir = new File(directory.getFile())
            dir.eachFileRecurse(FileType.FILES) { file ->
                if (file.getName().contains(".hpi")) {
                    String fileWithoutExt = file.name.take(file.name.lastIndexOf('.'))
                    def shortName = fileWithoutExt.take(fileWithoutExt.lastIndexOf('-'))
                    copyBundledPlugin(file.toURI().toURL(), shortName + ".jpi")
                    names.add(shortName)
                }
            }

            return names
        }

        @Override
        public PluginWrapper getPlugin(String shortName) {
            // load from our special dir?
            def superPlugin = super.getPlugin(shortName)
            if (superPlugin != null) {
                return superPlugin
            }

            // added breakpoint here but once it works it never gets here!
            return super.getPlugin(shortName)
        }
    }

I have a hunch that I can probably fix this with some gradle plugin copy task magic by just copying things under the shortName, but in the interim if that doesn't work, feel free to copy my rule.

Updated task

I was able to remove my hacked manager and just update my task to this, best of luck everyone

task resolveTestPlugins(type: Copy) {
    from configurations.testPlugins
    into new File(sourceSets.test.output.resourcesDir, 'test-dependencies')
    include '*.hpi'
    include '*.jpi'
    rename { filename ->
        // the plugin manager will load plugins by short name (trilead-api) instead of (trilead-api-1.0.8)
        String fileWithoutExt = filename.take(filename.lastIndexOf('.'))
        def shortName = fileWithoutExt.take(fileWithoutExt.lastIndexOf('-'))
        filename.replace fileWithoutExt, shortName
    }
    doLast {
        //
        def baseNames = source.collect { it.name[0..it.name.lastIndexOf('-') - 1] }
        new File(destinationDir, 'index').setText(baseNames.join('\n'), 'UTF-8')
    }
}

Double Edit

I think I might have commented on the wrong repo/issue :( since this isn't the plugin responsible for the testPlugins extension...

I could have sworn we copied that part from you at some point in history...... oh well, my 2 year memory points to this being where I copied it from: https://groups.google.com/forum/#!topic/job-dsl-plugin/Us5Ce1QHLVw

[basil]

I was able to work around this issue using a variant of TestDependenciesTask with javaConvention.sourceSets.test.output.resourcesDir changed to javaConvention.sourceSets.integrationTest.output.resourcesDir and the following code in my build.gradle file:

task resolveIntegrationTestDependencies(type: ResolveIntegrationTestDependenciesTask) {
  configuration = configurations.integrationTestRuntimeClasspath
}

tasks.processIntegrationTestResources.dependsOn resolveIntegrationTestDependencies

With this workaround in place, a build/resources/integrationTest/test-dependencies directory gets created and populated with ${artifactId}.hpi files and an index file that contains each artifactId. This eliminated my issue with recent versions of durable-task and workflow-durable-task-step.

I am hopeful that this issue will eventually be resolved upstream so that I can remove the workaround from my local build.

[AnEmortalKid]

@basil thanks for that, I was just looking at how to do something similar to my resolution (for an unrelated gradle project). Your hunch about the index and directory issue was on point.

[robons]

I was able to work around this issue using a variant of TestDependenciesTask with javaConvention.sourceSets.test.output.resourcesDir changed to javaConvention.sourceSets.integrationTest.output.resourcesDir and the following code in my build.gradle file:

task resolveIntegrationTestDependencies(type: ResolveIntegrationTestDependenciesTask) {
  configuration = configurations.integrationTestRuntimeClasspath
}

tasks.processIntegrationTestResources.dependsOn resolveIntegrationTestDependencies

With this workaround in place, a build/resources/integrationTest/test-dependencies directory gets created and populated with ${artifactId}.hpi files and an index file that contains each artifactId. This eliminated my issue with recent versions of durable-task and workflow-durable-task-step.

I am hopeful that this issue will eventually be resolved upstream so that I can remove the workaround from my local build.

Just thought I'd mention the changes I managed to make to my build.gradle.kts which took care of this without extending TestDependenciesTask directly (and incase anyone wants a quick copy-paste solution to the problem).

plugins {
    ...
    id("org.jenkins-ci.jpi") version "0.38.0" apply false
}

tasks {
    ...
    register<org.jenkinsci.gradle.plugins.jpi.TestDependenciesTask>("resolveIntegrationTestDependencies") {
        into {
            val javaConvention = project.convention.getPlugin<JavaPluginConvention>()
            File("${javaConvention.sourceSets.integrationTest.get().output.resourcesDir}/test-dependencies")
        }
        configuration = configurations.integrationTestRuntimeClasspath.get()
    }
    processIntegrationTestResources {
        dependsOn("resolveIntegrationTestDependencies")
    }
}

not that I condone using Kotlin.

[IppX]

@robons thanks, this helped a lot !

Here is the groovy version for anyone interested:

plugins {
    ...
    id("org.jenkins-ci.jpi") version "0.38.0" apply false
}

task resolveIntegrationTestDependencies(type: org.jenkinsci.gradle.plugins.jpi.TestDependenciesTask) {
  configuration = configurations.integrationTestRuntimeClasspath
  def javaConvention = project.convention.getPlugin(JavaPluginConvention)
  into file("${javaConvention.sourceSets.integrationTest.output.resourcesDir}/test-dependencies")
}

tasks.processIntegrationTestResources.dependsOn resolveIntegrationTestDependencies