search

jenkinsci / templating-engine-plugin

create tool-agnostic, templated pipelines to be shared by multiple teams
https://jenkinsci.github.io/templating-engine-plugin/latest/
Apache License 2.0
171 stars 59 forks source link

[Bug]: Failed to load existing library on v2.5.2 #298

Closed e88z4 closed 2 years ago

e88z4 commented 2 years ago

Jenkins Version

2.319.3

JTE Version

2.5.2

Bug Description

I have an existing JTE library that I have been developing with 2.3. Last night, we upgraded our non production environment to JTE v2.5.2 and tested our library.

The steps code library is below:

package libraries.pod_template.steps

import container.template.JnlpContainer
import container.template.SdkContainer
import container.template.DockerBuildContainer
import kubernetes.pod.volume.DockerBuildVolume
import kubernetes.Namespace
import kubernetes.Cloud
import kubernetes.pod.yamlPatch.DockerBuildContainerPatch

void call(Closure body){
    def cloud = config.cloud
    def containers = []
    def volumes = []
    def namespace = env.JENKINS_ENVIRONMENT == 'Prod'? Namespace.prod_jenkinsagents_ns : Namespace.stage_jenkinsagents_ns
    def yamlPatch = ''
    def nodeSelector = ['sunlife.com/node.licensing.none=true',
                        'mycompnay.com/node.network=corp',
                        'mycompany.com/node.sdlc.prod=true']

    containers << (new JnlpContainer()).create(config)

    if(config.build_agent != null){
        if(config.build_agent.sdk != null){
            containers << (new SdkContainer()).create(config)
        }

        if(config.build_agent.docker_build != null){
            if(config.build_agent.docker_build.enabled){
                containers << (new DockerBuildContainer().create(config))
                namespace = Namespace.prod_imagebuilderpipeline_ns
                cloud = Cloud.prod_whs_cloud
                yamlPatch = (new DockerBuildContainerPatch()).create()
            }
        }
    }

    podTemplate(
        cloud: cloud,
        containers: containers,
        volumes: volumes.flatten(),
        nodeSelector: nodeSelector.join(','),
        namespace: namespace,
        yaml: yamlPatch,
        yamlMergeStrategy: merge()
    ){
        node(POD_LABEL, body)
    }
}

The class source code is below:

package kubernetes.yamlPatch

public void create(){
    return '''
        apiVersion: v1
        kind: Pod
        spec:
          imagePullSecrets:
          - name: "dtr-secret"
          containers:
          - name: "image-build-sdk"
            env:
            - name: "DOCKER_CONTENT_TRUST_ROOT_PASSPHRASE"
              valueFrom:
                secretKeyRef:
                  key: "DOCKER_CONTENT_TRUST_ROOT_PASSPHRASE"
                  name: "dct-secret"
            - name: "DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE"
              valueFrom:
                secretKeyRef:
                  key: "DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE"
                  name: "dct-secret"
            - name: "CONTAINER_SUPPORT_SERVICE_URL"
              valueFrom:
                configMapKeyRef:
                  key: "CONTAINER_SUPPORT_SERVICE_URL"
                  name: "docker-image-build-configmap"
            - name: "DOCKER_LINTER_IMAGE"
              valueFrom:
                configMapKeyRef:
                  key: "DOCKER_LINTER_IMAGE"
                  name: "docker-image-build-configmap"
            - name: "DOCKER_CONTENT_TRUST_SIGNER_PUB_KEY"
              valueFrom:
                configMapKeyRef:
                  key: "DOCKER_CONTENT_TRUST_SIGNER_PUB_KEY"
                  name: "docker-image-build-configmap"
            - name: "DOCKER_CONFIG"
              value: "/dibv/.docker"
            securityContext:
              privileged: true
            volumeMounts:
            - mountPath: "/dibv"
              name: "docker-image-build-volume"
            - mountPath: "/signer-keys"
              name: "signer-pub-key-volume"
            - mountPath: "/dibv/.docker/trust/private/7e68336846014199f8dfabe09067b7933a7bef634ea74a8154c374f8da43d8a3.key"
              name: "dct-keys-volume"
              subPath: "7e68336846014199f8dfabe09067b7933a7bef634ea74a8154c374f8da43d8a3.key"
            - mountPath: "/dibv/.docker/trust/private/e0b2c3df2a30dbde2d99dd4752f50c45465dfc4175dc87c7bf45525c370a290c.key"
              name: "dct-keys-volume"
              subPath: "e0b2c3df2a30dbde2d99dd4752f50c45465dfc4175dc87c7bf45525c370a290c.key"
            - mountPath: "/etc/ucp-config.json"
              name: "ucp-config-secret-volume"
              readOnly: true
              subPath: "ucp-config.json"
          volumes:
          - name: "docker-image-build-volume"
            persistentVolumeClaim:
              claimName: "docker-image-build-pvc"
          - configMap:
              name: "signer-pub-key"
            name: "signer-pub-key-volume"
          - secret:
              defaultMode: 384
              secretName: "dct-keys-secret"
            name: "dct-keys-volume"
          - secret:
              secretName: "ucp-config-secret"
            name: "ucp-config-secret-volume"
    '''
}

Relevant log output

`[JTE] Failed to parse step text. Library: pod_template. Step: build_agent.
[JTE] 1: startup failed: 
[JTE] Compilation incomplete: expected to find the class kubernetes.pod.yamlPatch.DockerBuildContainerPatch in file:/var/jenkins_home/jobs/cloud-engineering/jobs/test-jte/branches/develop/builds/3/jte/src/kubernetes/pod/yamlPatch/DockerBuildContainerPatch.groovy, but the file contains the classes: kubernetes.yamlPatch.DockerBuildContainerPatch
[JTE] 
[JTE] 1 error
[JTE]  
[JTE] org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
[JTE] org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:958)
[JTE] org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:605)
[JTE] org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
[JTE] org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
[JTE] groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
[JTE] groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
[JTE] groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
[JTE] groovy.lang.GroovyShell.parse(GroovyShell.java:700)
[JTE] org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142)
[JTE] org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
[JTE] java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[JTE] java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[JTE] java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[JTE] java.base/java.lang.reflect.Method.invoke(Method.java:566)
[JTE] org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
[JTE] org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:71)
[JTE] org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
[JTE] org.boozallen.plugins.jte.init.primitives.injectors.StepWrapperFactory.prepareScript(StepWrapperFactory.groovy:158)
[JTE] org.boozallen.plugins.jte.init.primitives.injectors.StepWrapperFactory$prepareScript$0.callCurrent(Unknown Source)
[JTE] org.boozallen.plugins.jte.init.primitives.injectors.StepWrapperFactory.createFromFilePath(StepWrapperFactory.groovy:66)
[JTE] org.boozallen.plugins.jte.init.primitives.injectors.StepWrapperFactory$createFromFilePath.call(Unknown Source)
[JTE] org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
[JTE] org.boozallen.plugins.jte.init.primitives.injectors.StepWrapperFactory$createFromFilePath.call(Unknown Source)
[JTE] org.boozallen.plugins.jte.init.primitives.injectors.LibraryStepInjector$_injectPrimitives_closure3$_closure11.doCall(LibraryStepInjector.groovy:110)
[JTE] java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[JTE] java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[JTE] java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[JTE] java.base/java.lang.reflect.Method.invoke(Method.java:566)
[JTE] org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
[JTE] groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
[JTE] org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
[JTE] groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
[JTE] groovy.lang.Closure.call(Closure.java:414)
[JTE] groovy.lang.Closure.call(Closure.java:430)
[JTE] org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2040)
[JTE] org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:1895)
[JTE] org.codehaus.groovy.runtime.dgm$160.invoke(Unknown Source)
[JTE] org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274)
[JTE] org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)
[JTE] org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
[JTE] org.boozallen.plugins.jte.init.primitives.injectors.LibraryStepInjector$_injectPrimitives_closure3.doCall(LibraryStepInjector.groovy:109)
[JTE] java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[JTE] java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[JTE] java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[JTE] java.base/java.lang.reflect.Method.invoke(Method.java:566)
[JTE] org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
[JTE] groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
[JTE] org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
[JTE] groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
[JTE] groovy.lang.Closure.call(Closure.java:414)
[JTE] org.codehaus.groovy.runtime.DefaultGroovyMethods.callClosureForMapEntry(DefaultGroovyMethods.java:5276)
[JTE] org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2117)
[JTE] org.codehaus.groovy.runtime.dgm$164.invoke(Unknown Source)
[JTE] org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoMetaMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:251)
[JTE] org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:71)
[JTE] org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
[JTE] org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
[JTE] org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
[JTE] org.boozallen.plugins.jte.init.primitives.injectors.LibraryStepInjector.injectPrimitives(LibraryStepInjector.groovy:105)
[JTE] java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[JTE] java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[JTE] java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[JTE] java.base/java.lang.reflect.Method.invoke(Method.java:566)
[JTE] org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
[JTE] groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
[JTE] groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
[JTE] groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
[JTE] groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:810)
[JTE] org.boozallen.plugins.jte.init.primitives.TemplatePrimitiveInjector.invokeMethod(TemplatePrimitiveInjector.groovy)
[JTE] groovy.lang.GroovyObject$invokeMethod.call(Unknown Source)
[JTE] org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
[JTE] groovy.lang.GroovyObject$invokeMethod.call(Unknown Source)
[JTE] org.boozallen.plugins.jte.init.primitives.TemplatePrimitiveInjector$_invoke_closure1.doCall(TemplatePrimitiveInjector.groovy:149)
[JTE] java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[JTE] java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[JTE] java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[JTE] java.base/java.lang.reflect.Method.invoke(Method.java:566)
[JTE] org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
[JTE] groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
[JTE] org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:294)
[JTE] groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1022)
[JTE] groovy.lang.Closure.call(Closure.java:414)
[JTE] groovy.lang.Closure.call(Closure.java:430)
[JTE] org.codehaus.groovy.runtime.DefaultGroovyMethods.each(DefaultGroovyMethods.java:2040)
[JTE] org.codehaus.groovy.runtime.dgm$162.invoke(Unknown Source)
[JTE] org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite$PojoMetaMethodSiteNoUnwrapNoCoerce.invoke(PojoMetaMethodSite.java:274)
[JTE] org.codehaus.groovy.runtime.callsite.PojoMetaMethodSite.call(PojoMetaMethodSite.java:56)
[JTE] org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125)
[JTE] org.boozallen.plugins.jte.init.primitives.TemplatePrimitiveInjector.invoke(TemplatePrimitiveInjector.groovy:144)
[JTE] java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[JTE] java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[JTE] java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[JTE] java.base/java.lang.reflect.Method.invoke(Method.java:566)
[JTE] org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
[JTE] groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
[JTE] org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.invoke(StaticMetaMethodSite.java:46)
[JTE] org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.callStatic(StaticMetaMethodSite.java:102)
[JTE] org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
[JTE] org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
[JTE] org.boozallen.plugins.jte.init.primitives.TemplatePrimitiveInjector.orchestrate(TemplatePrimitiveInjector.groovy:92)
[JTE] org.boozallen.plugins.jte.init.primitives.TemplatePrimitiveInjector$orchestrate.call(Unknown Source)
[JTE] org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
[JTE] org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
[JTE] org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
[JTE] org.boozallen.plugins.jte.job.TemplateFlowDefinition.create(TemplateFlowDefinition.groovy:85)
[JTE] org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:310)
[JTE] hudson.model.ResourceController.execute(ResourceController.java:99)
[JTE] hudson.model.Executor.run(Executor.java:432)
org.boozallen.plugins.jte.util.AggregateException: JTE Pipeline Initialization failed during injectPrimitives
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
    at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:83)
    at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:105)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:239)
    at org.boozallen.plugins.jte.init.primitives.TemplatePrimitiveInjector.invoke(TemplatePrimitiveInjector.groovy:143)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
    at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
    at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.invoke(StaticMetaMethodSite.java:46)
    at org.codehaus.groovy.runtime.callsite.StaticMetaMethodSite.callStatic(StaticMetaMethodSite.java:102)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
    at org.boozallen.plugins.jte.init.primitives.TemplatePrimitiveInjector.orchestrate(TemplatePrimitiveInjector.groovy:92)
    at org.boozallen.plugins.jte.init.primitives.TemplatePrimitiveInjector$orchestrate.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
    at org.boozallen.plugins.jte.job.TemplateFlowDefinition.create(TemplateFlowDefinition.groovy:85)
    at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:310)
    at hudson.model.ResourceController.execute(ResourceController.java:99)
    at hudson.model.Executor.run(Executor.java:432)
Finished: FAILURE`

Steps to Reproduce

Execute pipeline with the new JTE version v2.5.2

e88z4 commented 2 years ago

I found the issue. The package declaration package kubernetes.yamlPatch doesn't match with the class import declaration in the steps script import import kubernetes.pod.yamlPatch.DockerBuildContainerPatch

This was previously ignored with JTE v2.3.0 but it is being considered as an error with JTE v2.5.2.

steven-terrana commented 2 years ago

would you be able to provide a minimally reproducible example in a git repo?

The step / class contents don't matter much - just the import statements and library sources with the same directory structure.

right off the bat - i see a couple head scratchers.

  1. the class source file you provided isn't a class?
  2. the package name in the class source text you provided doesnt line up with the package name you're importing?

because you're importing import kubernetes.pod.yamlPatch.DockerBuildContainerPatch I expected to see

// current: package kubernetes.yamlPatch
package kubernetes.pod.yamlPatch

class DockerBuildContainerPatch{}

the error seems to be saying the same thing

Compilation incomplete: expected to find the class kubernetes.pod.yamlPatch.DockerBuildContainerPatch in *, but the file contains the classes: kubernetes.yamlPatch.DockerBuildContainerPatch

try changing the package name to what it should be.

the bigger question is why this ever worked.

steven-terrana commented 2 years ago

alrighty - you beat me to it :)