search

ExpediaGroup / jenkins-spock

Unit-test Jenkins pipeline code with Spock
https://javadoc.io/doc/com.homeaway.devtools.jenkins/jenkins-spock
Apache License 2.0
187 stars 76 forks source link

java.lang.IllegalStateException: Jenkins.instance #72

Closed mrnguyen4386 closed 4 years ago

mrnguyen4386 commented 4 years ago

I ran into the following issue:

java.lang.IllegalStateException: Jenkins.instance is missing. Read the documentation of Jenkins.getInstanceOrNull to see what you are doing wrong.
        at jenkins.model.Jenkins.get(Jenkins.java:778)
        at hudson.model.Descriptor.getConfigFile(Descriptor.java:904)
        at hudson.model.Descriptor.load(Descriptor.java:892)
        at org.jenkinsci.plugins.pipeline.milestone.MilestoneStep$DescriptorImpl.load(MilestoneStep.java:105)
        at org.jenkinsci.plugins.pipeline.milestone.MilestoneStep$DescriptorImpl.<init>(MilestoneStep.java:90)
        at java.lang.Class.newInstance(Class.java:442)
        at com.homeaway.devtools.jenkins.testing.APipelineExtensionDetector.getPipelineSteps(APipelineExtensionDetector.java:83)
        at com.homeaway.devtools.jenkins.testing.JenkinsPipelineSpecification.setupSpec(JenkinsPipelineSpecification.groovy:1005)
awittha commented 4 years ago

What code did you write and run to cause that?

mrnguyen4386 commented 4 years ago

Sorry, I am super new at this. Thank you for your patience.

I updated my pom.xml to have the following:

<parent>
      <groupId>org.jenkins-ci.plugins</groupId>
      <artifactId>plugin</artifactId>
      <version>4.2</version>
      <relativePath />
    </parent>

and I ran this test that ran into the issue:


class CiBuildTriggersHelperSpec extends JenkinsPipelineSpecification {

    def ciBuildTriggersHelper = null

    def setup(){
        ciBuildTriggersHelper = loadPipelineScriptForTest('vars/ciBuildTriggersApp.groovy')
    }
mrnguyen4386 commented 4 years ago

pom.xml.zip

awittha commented 4 years ago

Hm; I'd hope to be able to see the implementation of ciBuildTriggersApp.groovy, but I understand if that's not possible.

Jenkins is already Mocked!

It looks like somewhere there's probably a line like this:

import jenkins.model.Jenkins

...

Jenkins.get()

This is a static method on the jenkins.model.Jenkins class, which normally returns the current instance of Jenkins. But, during unit tests, there is no real instance of Jenkins.

Per the documentation:

Mock Jenkins There is a Spock Mock of type Jenkins available at getPipelineMock("Jenkins"). It is set up as follows:

It looks like Jenkins.get() should've been automatically stubbed to return the existing mock, as well.

mrnguyen4386 commented 4 years ago

@awittha Thank you for the quick response. I don't know if this make a difference, but the unit test ran without issue before I modified the pom.xml to have org.jenkins-ci.plugins as parent and added some dependencies to comply upper bound dependency issued by enforce plugin. Additionally, ciBuildTriggersApp.groovy doesn't import jenkins.model.Jenkins or doesn't used that object explicitly.

awittha commented 4 years ago

Before updating your pom.xml parent...

Was your parent pom an older version of the Jenkins plugin parent pom? That is to say, is your project a Jenkins plugin?

Or, was your parent pom something else?

mrnguyen4386 commented 4 years ago

I didn't have a parent in the older pom.xml. Below is the pom.xml content that run unit test without issue without parent. I fear the way pom.xml is configured may be the cause for the issue? 

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.intusurg.spr.jenkins</groupId>
    <artifactId>jenkins-shared-lib</artifactId>
    <packaging>jar</packaging>
    <version>0.4.0</version>
    <description>Jenkins Shared Library</description>

    <repositories>
        <repository>
            <id>jenkins-releases</id>
            <name>Jenkins Releases</name>
            <url>http://repo.jenkins-ci.org/releases</url>
        </repository>
    </repositories>

    <properties>
        <groovy.core.version>2.4.11</groovy.core.version>
        <groovy.gmaven.pluginVersion>1.6.1</groovy.gmaven.pluginVersion>
        <google.guava.version>20.0</google.guava.version>

        <jenkins-spock.version>2.0.0</jenkins-spock.version>
        <jenkins.version>2.102</jenkins.version>
        <jenkins.servlet.version>3.1.0</jenkins.servlet.version>
        <jenkins.workflow.cps.version>2.36</jenkins.workflow.cps.version>
        <jenkins.workflow.step.version>2.10</jenkins.workflow.step.version>
        <jenkins.workflow.basic.steps.version>2.6</jenkins.workflow.basic.steps.version>
        <jenkins.workflow.durable.task.steps.version>2.21</jenkins.workflow.durable.task.steps.version>
        <jenkins.pipeline.stage.step.version>2.3</jenkins.pipeline.stage.step.version>
        <jenkins.git.version>3.9.1</jenkins.git.version>

        <junit.version>4.12</junit.version>
        <junit.plugin.version>1.24</junit.plugin.version>
        <pipeline-utility-steps>2.3.1</pipeline-utility-steps>
        <surefire.pluginVersion>2.22.0</surefire.pluginVersion>

        <logback.configration>logback-test.xml</logback.configration>
        <logdir>${project.build.directory}/log</logdir>
        <test.loglevel>ERROR</test.loglevel>
        <log.logback.version>1.2.3</log.logback.version>
        <log.slf4j.version>1.7.25</log.slf4j.version>
        <build-user-vars-plugin.version>1.5</build-user-vars-plugin.version>

        <sonar.host.url>https://sonarqube-dev.corp.intusurg.com</sonar.host.url>
        <sonar.projectName>Jenkins Shared Library</sonar.projectName>
        <sonar.projectKey>jenkins-shared-lib</sonar.projectKey>
        <sonar.sources>src</sonar.sources>
        <sonar.tests>test</sonar.tests>
        <sonar.groovy.jacoco.reportPath>target/jacoco.exec</sonar.groovy.jacoco.reportPath>
    </properties>
    <dependencyManagement>
        <dependencies>
            <!-- Transitive Dependencies that need Managing -->
            <dependency>
                <groupId>com.google.guava</groupId>
                <artifactId>guava</artifactId>
                <version>${google.guava.version}</version>
            </dependency>
        </dependencies>
    </dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>org.apache.ivy</groupId>
        <artifactId>ivy</artifactId>
        <version>2.4.0</version>
      </dependency>
        <dependency>
            <groupId>com.launchdarkly</groupId>
            <artifactId>launchdarkly-java-server-sdk</artifactId>
            <version>4.11.1</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>com.homeaway.devtools.jenkins</groupId>
            <artifactId>jenkins-spock</artifactId>
            <version>${jenkins-spock.version}</version>
            <scope>test</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.jenkins-ci.plugins/build-user-vars-plugin -->
        <dependency>
          <groupId>org.jenkins-ci.plugins</groupId>
          <artifactId>build-user-vars-plugin</artifactId>
          <version>${build-user-vars-plugin.version}</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.jenkins-ci.plugins/ssh-credentials -->
        <dependency>
          <groupId>org.jenkins-ci.plugins</groupId>
          <artifactId>ssh-credentials</artifactId>
          <version>1.18</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.jenkins-ci.plugins/credentials-binding -->
        <dependency>
          <groupId>org.jenkins-ci.plugins</groupId>
          <artifactId>credentials-binding</artifactId>
          <version>1.23</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/com.cloudbees/groovy-cps -->
        <dependency>
          <groupId>com.cloudbees</groupId>
          <artifactId>groovy-cps</artifactId>
          <version>1.31</version>
        </dependency>
        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-core</artifactId>
            <version>${log.logback.version}</version>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-classic</artifactId>
            <version>${log.logback.version}</version>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>${junit.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.jenkins-ci.main</groupId>
            <artifactId>jenkins-core</artifactId>
            <version>${jenkins.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.jenkins-ci.plugins.workflow</groupId>
            <artifactId>workflow-basic-steps</artifactId>
            <version>${jenkins.workflow.basic.steps.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.jenkins-ci.plugins.workflow</groupId>
            <artifactId>workflow-cps</artifactId>
            <version>${jenkins.workflow.cps.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <!-- provides sh() step -->
            <groupId>org.jenkins-ci.plugins.workflow</groupId>
            <artifactId>workflow-durable-task-step</artifactId>
            <version>${jenkins.workflow.durable.task.steps.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <!-- provides stage() step -->
            <groupId>org.jenkins-ci.plugins</groupId>
            <artifactId>pipeline-stage-step</artifactId>
            <version>${jenkins.pipeline.stage.step.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <!-- provides git() step -->
            <groupId>org.jenkins-ci.plugins</groupId>
            <artifactId>git</artifactId>
            <version>${jenkins.git.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>javax.servlet-api</artifactId>
            <version>${jenkins.servlet.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.jenkins-ci.plugins</groupId>
            <artifactId>junit</artifactId>
            <version>${junit.plugin.version}</version>
        </dependency>
        <dependency>
          <groupId>org.jenkins-ci.plugins</groupId>
          <artifactId>pipeline-utility-steps</artifactId>
          <version>${pipeline-utility-steps}</version>
        </dependency>
        <dependency>
            <groupId>org.codehaus.groovy</groupId>
            <artifactId>groovy-all</artifactId>
            <version>${groovy.core.version}</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>jcl-over-slf4j</artifactId>
            <version>${log.slf4j.version}</version>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>log4j-over-slf4j</artifactId>
            <version>${log.slf4j.version}</version>
            <scope>runtime</scope>
        </dependency>
    </dependencies>
    <build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>${surefire.pluginVersion}</version>
                    <executions>
                        <execution>
                            <id>default-test</id>
                            <goals>
                                <goal>test</goal>
                            </goals>
                            <configuration>
                                <forkCount>1</forkCount>
                                <includes>
                                    <include>${test.pattern}</include>
                                </includes>
                                <useManifestOnlyJar>false</useManifestOnlyJar>
                                <systemPropertyVariables>
                                    <root.loglevel>${test.loglevel}</root.loglevel>
                                    <root.appender>Stdout</root.appender>
                                    <test.loglevel>${test.loglevel}</test.loglevel>
                                    <logdir>${logdir}</logdir>
                                </systemPropertyVariables>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
                <plugin>
                    <groupId>org.codehaus.gmavenplus</groupId>
                    <artifactId>gmavenplus-plugin</artifactId>
                    <version>${groovy.gmaven.pluginVersion}</version>
                    <executions>
                        <execution>
                            <id>groovy</id>
                            <goals>
                                <goal>addSources</goal>
                                <goal>addTestSources</goal>
                                <goal>generateStubs</goal>
                                <goal>generateTestStubs</goal>
                                <goal>compile</goal>
                                <goal>compileTests</goal>
                                <goal>removeStubs</goal>
                                <goal>removeTestStubs</goal>
                            </goals>
                            <configuration>
                                <configScript>config.groovy</configScript>
                                <sources>
                                    <source>
                                        <directory>src</directory>
                                        <includes>
                                            <include>**/*.groovy</include>
                                        </includes>
                                    </source>
                                </sources>
                                <testSources>
                                    <testSource>
                                        <directory>test</directory>
                                        <includes>
                                            <include>**/*.groovy</include>
                                        </includes>
                                    </testSource>
                                </testSources>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
                <plugin>
                    <groupId>org.jacoco</groupId>
                    <artifactId>jacoco-maven-plugin</artifactId>
                    <version>0.8.3</version>
                    <executions>
                        <execution>
                            <goals>
                                <goal>prepare-agent</goal>
                            </goals>
                        </execution>
                        <execution>
                            <id>report</id>
                            <phase>test</phase>
                            <goals>
                                <goal>report</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </pluginManagement>
        <plugins>
            <plugin>
                <groupId>org.codehaus.gmavenplus</groupId>
                <artifactId>gmavenplus-plugin</artifactId>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
            </plugin>
            <plugin>
                <groupId>org.jacoco</groupId>
                <artifactId>jacoco-maven-plugin</artifactId>
            </plugin>
        </plugins>
        <testResources>
            <testResource>
                <includes>
                    <include>vars/**/*.groovy</include>
                </includes>
                <directory>${project.basedir}</directory>
            </testResource>
            <testResource>
                <directory>test/resources</directory>
            </testResource>
            <testResource>
                <directory>resources</directory>
            </testResource>
        </testResources>
    </build>
</project>
awittha commented 4 years ago

Is your project a Jenkins plugin? Generally, I'd say you shouldn't use the Jenkins plugin parent pom, if your project is not a Jenkins plugin.

awittha commented 4 years ago

ciBuildTriggersApp.groovy doesn't import jenkins.model.Jenkins or doesn't used that object explicitly.

Yeah, it's coming from here: https://github.com/jenkinsci/pipeline-milestone-step-plugin/blob/master/src/main/java/org/jenkinsci/plugins/pipeline/milestone/MilestoneStep.java#L105

when MilestoneStep's Descriptor is instantiated (as must be done by jenkins-spock in order to get the name of the step and set up the correct mock), MilestoneStep tries to load configuration from Jenkins... but there is no Jenkins. The specific path it uses hits Jenkins.get(), which isn't stubbed like all of the other Jenkins.get....() methods.

I don't know why, exactly, the parent pom.xml change caused this to surface, but if I had to guess I'd guess that adding the parent pom (or some of the other pom changes you mentioned) brought the Milestone plugin onto the classpath, causing jenkins-spock to try to work with it.

You'd have gotten a different error - one that you could maybe work around by adding stubbing to the Jenkins mock - if Jenkins.get() had been properly stubbed by jenkins-spock.

mrnguyen4386 commented 4 years ago

That makes sense. Thank you very much for the explanation. Just want to clarify, as of now there is no work around beside not having the parent pom.xml because Jenkins wasn't stubbed properly by jenkins-spock?

awittha commented 4 years ago

as of now there is no work around beside not having the parent pom.xml because Jenkins wasn't stubbed properly by jenkins-spock?

Actually...

I misspoke earlier. This is happening before any Specification-specific stuff happens... this is happening during static setup of the test-suite, before any mocks exist.

There will never be a mock Jenkins at that time. This makes sense, because looking closer at the code, this actually would have worked if it had happened after the mock Jenkins was created by jenkins-spock.

awittha commented 4 years ago

The stack trace starts here: https://github.com/ExpediaGroup/jenkins-spock/blob/ebdaf89887cce6552d52349af7b703e365b76430/src/main/groovy/com/homeaway/devtools/jenkins/testing/JenkinsPipelineSpecification.groovy#L1005

in static setup before the JenkinsPipelineSpecification runs any tests or creates any mocks - it's still identifying what it will need to mock.

That leads to it instantiating MilestoneStep.DescriptorImpl (so that it can call getFunctionNamed() and know what to name the mock) and running into the load() step from here: https://github.com/jenkinsci/pipeline-milestone-step-plugin/blob/master/src/main/java/org/jenkinsci/plugins/pipeline/milestone/MilestoneStep.java#L105

the implementation of which unfortunately calls Jenkins.get().

There is, indeed, no real Jenkins and no mock Jenkins at this time.

There cannot be, either... because this is static setup, not specific to any test case. A mock created and used here will either be inaccessible during any test or shared across all tests.

I am not sure how to fix this, anymore. I'll think about it.

awittha commented 4 years ago

In this case, it's extra-tough because MilestoneStep.DescriptorImpl is a static final class. We probably cannot mock or stub it or otherwise modify its behavior, so it's probably always going to try to load().

This kind of descriptor/extension-instantiation- or classload-time (truly static) Jenkins-dependent setup is very frustrating for offline unit-tests; Milestone is far from the only bit of code to to this.

Maybe a special mock Jenkins created and used only at static setup-time just to placate these kinds of setup?

But even then, that won't work properly because there is no way for you, the test author, to access that mock and stub it to do what the step expects. How can you stub the implementation of load() so that Milestone is happy? Where would you write that test code? In setupSpec in your own Specification? Well... the parent setupSpec is run first, per Spock's design, so your own method is "too late." But of course, if your code could run before the parent's setupSpec, the mock Jenkins wouldn't be around for you to stub!

awittha commented 4 years ago

FOR NOW, just exclude the Milestone jenkins plugin from your project if you can.

If your code-under-test actually uses it... then I don't know what to tell you.

mrnguyen4386 commented 4 years ago

@awittha I will exclude Milestone jenkins plugin from my project since none of my code under test is not using it. I think the parent org.jenkins-ci.plugins pom.xml has the milestone plugin. Do you know a way to exclude it from the parent pom.xml or it's something I have to not have the parent and add dependency as needed? Again thank you very much for your support.

awittha commented 4 years ago

If the parent pom has it as a direct <dependency>, there's nothing you can do from a child pom.

If the parent pom has some dependency that has the Milestone plugin as a transitive dependency, you need a Maven exclusion for the Milestone plugin, placed in the declaration of whichever dependency(ies) are depending on the Milestone plugin.

You can run mvn dependency:tree and search for "milestone" to locate the plugin(s) that are bringing it in.

awittha commented 4 years ago

The core problem should be resolved in jenkins-spock:2.1.4.

Depending on the situation, there are still several ways to resolve the original exception. In this case, I believe excluding the Milestone plugin (if your code actually doesn't use it) is still the right choice.

Other options are now detailed in the documentation. See https://github.com/ExpediaGroup/jenkins-spock/blob/master/src/test/groovy/com/homeaway/devtools/jenkins/testing/DescriptorTimeJenkinsInteractingSpec.groovy for an example of how you might handle the situation where you actually needed the milestone step.