Problems when trying out MPL

[jfischer1]

Hi, when trying out MPL I stumbled over two things. Don't know if these are bugs, it is more likely that I configured/used something wrong. Can you help please?

Setup: Jenkins test instance without any slave nodes, running with plugin 'Pipeline: Groovy 2.83' (contains org.jenkins-ci.plugins.workflow:workflow-cps:2.83 and com.cloudbees:groovy-cps:1.32). Using MPL branch master, commit 1ec0f57.

Problem 1: when running MPLPipeline with default configuration, the following exception occurs:

hudson.remoting.ProxyException: groovy.lang.MissingMethodException: No signature of method: java.util.ArrayList.join() is applicable for argument types: () values: []
Possible solutions: join(java.lang.String), min(), min(groovy.lang.Closure), min(java.util.Comparator), wait(), find()
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:153)
    at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:161)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:165)
    at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
    at MPLModule.call(MPLModule.groovy:60)
    at MPLModule.call(MPLModule.groovy)
    at MPLPipeline.call(MPLPipeline.groovy:52)

Changing MPLModule.groovy@60 to def module_path = "modules/${base.last()}/${base.join(**''**)}.groovy" seems to fix it. But this seems strange to me. It looks like in Groovy there is no method with signature join(void) for lists. If this were true, how could the original line have ever worked? Did I miss something?

Problem 2: I fixed problem 1 with the modification above and run MPLPipeline with default configuration. There, the following exception occurs:

hudson.remoting.ProxyException: groovy.lang.MissingMethodException: No signature of method: org.jenkinsci.plugins.workflow.cps.CpsThread.getContextVariable() is applicable for argument types: (java.lang.Class, org.kohsuke.groovy.sandbox.impl.SandboxedMethodClosure, org.kohsuke.groovy.sandbox.impl.SandboxedMethodClosure) values: [class hudson.FilePath, org.kohsuke.groovy.sandbox.impl.SandboxedMethodClosure@523cd0e8, ...]
Possible solutions: getContextVariables()
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:153)
    at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:161)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:165)
    at org.kohsuke.groovy.sandbox.impl.Checker$checkedCall$3.callStatic(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
    at com.griddynamics.devops.mpl.Helper.pathExists(Helper.groovy:148)
    at com.griddynamics.devops.mpl.Helper$pathExists$2.callStatic(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallStatic(CallSiteArray.java:56)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callStatic(AbstractCallSite.java:194)
    at org.kohsuke.groovy.sandbox.impl.Checker$2.call(Checker.java:194)
    at org.kohsuke.groovy.sandbox.GroovyInterceptor.onStaticCall(GroovyInterceptor.java:35)
    at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onStaticCall(SandboxInterceptor.java:186)
    at org.kohsuke.groovy.sandbox.impl.Checker$2.call(Checker.java:192)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedStaticCall(Checker.java:196)
    at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:103)
    at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:17)
    at MPLModule.call(MPLModule.groovy:66)
    at MPLModule.call(MPLModule.groovy)
    at MPLPipeline.call(MPLPipeline.groovy:52)

If this method is not found by Groovy runtime, then maybe I'm using the wrong cps plugin? However, I double checked: I'm using workflow-cps:2.83 and according to source code at github, getContextVariable with matching signature should be available. Can you give me advice what I'm missing?

[sparshev]

Hi @jfischer1 , so the MPLPipeline to build MPL itself is running daily on the LTS & Latest jenkins (and plugins of course) - you can check it here: https://app.circleci.com/pipelines/github/griddynamics/mpl?branch=master - so I see no way it's not working correctly.

Unfortunately for now I can't get your setup from "MPLPipeline with default configuration" - not sure what you're trying to do and how are you running the thing. Can recommend to start slow with the basic example of building the MPL itself and I think it will be smooth: https://github.com/griddynamics/mpl/wiki/MPL-Build-example

When it will work fine for you, you can switch to the more complicated examples here: https://github.com/griddynamics/mpl/wiki#examples and play with the configured environment further.

[jfischer1]

Hi @sparshev, my pipeline setup is quite simple:

MPLPipeline{}

I know that your project's CI jobs build through, and therefore I don't think that the problems I described are bugs in your code. When trying to get MPL run on my system, I got stuck and my hope was that you just say 'Hey, I know this error message. Cause is ...'. So I'll have a look at MPL's CI configuration and try to spot a difference to my configuration. Thank you for the quick reply.

[sparshev]

Yeah, basically to help you I need to know what is the differences you have in your environment. Jenkins is quite flexible system, so it's hard to predict how are you using the MPL for your projects. So if you can describe it in detail - I will be able to help you. But I hope the examples will help in much effective way)

[jfischer1]

I have to admit that I'm not very experienced with problems so deep in the Groovy-part of Jenkins. What kind of additional information do you need? Maybe the list+version of installed plugins? Regarding the Jenkins instance where the issue occurred: it's a Jenkins in a VM for testing. No slaves, just one master running on Ubuntu Linux.

[sparshev]

Hi, if you can describe what you're trying to do - that will be great. What kind of project you're trying to build, how the shared libs are configured, what kind of changes you've made to mpl and so on.

And yhay will be great to hear if the basic mpl build example is worked for you or not.

[jfischer1]

Hi, most of the description what I'm trying to do is already in the comments above. I'll try to summarize it now.

What do I want to do? Create a pipeline for multi-target-platform parallel C++ builds.

For this, I set up a Jenkins in a VM without any slave nodes without any slave nodes, running with plugin 'Pipeline: Groovy 2.83' (contains org.jenkins-ci.plugins.workflow:workflow-cps:2.83 and com.cloudbees:groovy-cps:1.32). Using MPL branch master, commit 1ec0f57.

Initial Jenkinsfile from project looks like this:

def projectParameters = readYaml text: '''
pipelinelibs:
  - identifier: mpl@master
    remote: /home/my-user-name/mpl
'''
projectParameters.pipelinelibs.each { lib -> 
  library(
    identifier: lib.identifier,
    retriever: modernSCM([
        $class: 'GitSCMSource',
        credentialsId: some-credentials-id,
        remote: lib.remote
    ]))
}
MPLPipeline{}

I didn't install tools like maven so I would expect pipeline to fail due to missing executables. Here comes the strange part: at first this happened as expected. Then I started to write my own nested pipeline library. At some point, problem 1 (see my initial post) showed up:

hudson.remoting.ProxyException: groovy.lang.MissingMethodException: No signature of method: java.util.ArrayList.join()

I started to revert my changes until I was at the initial Jenkinsfile described in this comment. Problem persisted. Therefore, I tried to fix this join() call my modifying MPLModule.groovy@60 to

def module_path = "modules/${base.last()}/${base.join('')}.groovy"

This helped in the sense that MPL proceeded further, but got finally stuck with problem 2:

hudson.remoting.ProxyException: groovy.lang.MissingMethodException: No signature of method: org.jenkinsci.plugins.workflow.cps.CpsThread.getContextVariable() is applicable for argument types: (java.lang.Class, org.kohsuke.groovy.sandbox.impl.SandboxedMethodClosure, org.kohsuke.groovy.sandbox.impl.SandboxedMethodClosure) values: [class hudson.FilePath, org.kohsuke.groovy.sandbox.impl.SandboxedMethodClosure@523cd0e8, ...]
Possible solutions: getContextVariables()

What is really weird is that it worked for some time. I'm definitely missing something.

[sparshev]

Hi @jfischer1 - that's exactly why I asking for the details. For sure using library() is not so simple as just call MPLPipeline{} you mentioned above.

The MPL library is using builtin jenkins systems to simplify the structure of the pipelines and that means you can't use such dynamic loading as library(). You have to use global shared libraries configuration to specify mpl library (how described on the wiki page here) - that will allow MPL to use otherwise restricted methods by specify @Library('mpl') _ as in the Jenkinsfile of MPL.

library() step don't allow to execute MPL pipeline logic as trusted one, so there is no way to use this call (I think you don't want to execute your pipeline logic with no sandbox flag, right?). The MPL itself manages to run its modules in sandbox and steps as trusted, so no worries here - you just need to make sure that no one will be able to change the shared lib logic.

[jfischer1]

Hi @spashev, this explains a lot. Now that you mention it, I remember to use @Library at first for some time. Maybe that's the reason why 'suddenly' a lot of approvals were needed. Thank you for your help.

[sparshev]

Sure thing, any time)