Jenkins complains about CPS mismatch

[sdkrach]

We are currently evaluating this library as foundation for modularizing our Jenkins build pipelines. We defined a pipeline and stages similar to the MPLPipeline (https://github.com/MDSD-Tools/Build-JenkinsLibrary/tree/mdsdmodular). The pipeline itself works, the modules are called in the right order and the build succeeds for our migrated test project (https://github.com/MDSD-Tools/ThirdParty-Library/tree/modularPipeline).

Unfortunately, we see the following errors in the jenkins build log: expected to call org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.evaluate but wound up catching Prepare.run; see: https://jenkins.io/redirect/pipeline-cps-method-mismatches/ (https://build.mdsd.tools/job/MDSD-Tools/job/ThirdParty-Library/job/modularPipeline/2/console) Prepare is a module of our pipeline (in fact a hierarchy of some modules), which is executed fine. Still, the error is logged for every module which is called by the pipeline or by other modules.

While the pipeline is working, we suspect Jenkins' CPS transformation is failing. We get the same log messages when running the pipeline of the MPL master branch. We configured Jenkins (LTS 2.176.1) to use the MPL library from your Github repository.

Is this a know bug? Are we maybe missing some Jenkins configuration? Are you aware of potential incompatibilities with recent Jenkins versions?

[sparshev]

Hi @sdkrach ,

We using the same version 2.176.1 and see no such issues.

So let me collect some info - could you please provide the list of the plugins (and versions) you using on the Jenkins?

Thank you

[sparshev]

I see you using pipeline definition inside the pipeline, quite sure we did not tested such approach - that could be a part of the issue...

[sparshev]

Also I see you use custom ConfigurationManager - probably #10 and #27 (with implementation of MPLConfig object) will help to understand the issue.

[sparshev]

Overall - since your lib contains quite complicated groovy logic, there is a huge chance it have some CPS issue... I will recommend to localize, where the issue is coming from (you can use println - they will be visible in the Jenkins master log even if they are executed from the class function).

Hopefully it's not a critical one and you will be able to fix it. Sometimes just a simple @NonCPS will help with some function, but it's better to check where it's coming from.

Overall recommendation - be really careful with creating some custom classes, they could produce a hidden issues (exceptions will not be shown in the Jenkins job or master logs).

[sdkrach]

Hi @sparshev,

thank you for your very fast and elaborate response! I cloned the MPL library into our repository. We are getting the same error messages without using our extended configuration management for the build job of the MPL Library (https://build.mdsd.tools/job/MDSD-Tools/job/mpl/job/master/3/console). The build understandably fails as we do not have the Maven 3 tool configured. Still, the log shows the same error messages for the succeeding Checkout and DefaultCheckout modules.

We updated all our plugins to the most recent version. As requested I list the versions which we are using:

Plugin	Version	Activated
ace-editor	1.1	true
analysis-model-api	5.2.0	true
antisamy-markup-formatter	1.5	true
apache-httpcomponents-client-4-api	4.5.5-3.0	true
authentication-tokens	1.3	true
authorize-project	1.3.0	true
basic-branch-build-strategies	1.3.2	true
bouncycastle-api	2.17	true
branch-api	2.5.3	true
cloud-stats	0.24	true
cloudbees-folder	6.9	true
command-launcher	1.3	true
config-file-provider	3.6.2	true
credentials	2.2.0	true
credentials-binding	1.19	true
display-url-api	2.3.1	true
docker-commons	1.15	true
docker-workflow	1.18	true
durable-task	1.30	true
email-ext	2.66	true
embeddable-build-status	2.0.2	true
external-monitor-job	1.7	true
git	3.10.1	true
git-client	2.8.0	true
git-server	1.7	true
github	1.29.4	true
github-api	1.95	true
github-branch-source	2.5.4	true
github-oauth	0.32	true
handlebars	1.1.1	true
htmlpublisher	1.18	true
jackson2-api	2.9.9.1	true
jacoco	3.0.4	true
jdk-tool	1.3	true
jquery-detached	1.2.1	true
jsch	0.1.55	true
junit	1.28	true
ldap	1.20	true
lockable-resources	2.5	true
mailer	1.23	true
mask-passwords	2.12.0	true
matrix-auth	2.4.2	true
matrix-project	1.14	true
momentjs	1.1.1	true
pam-auth	1.5.1	true
pipeline-build-step	2.9	true
pipeline-graph-analysis	1.10	true
pipeline-input-step	2.10	true
pipeline-milestone-step	1.3.1	true
pipeline-model-api	1.3.9	true
pipeline-model-declarative-agent	1.1.1	true
pipeline-model-definition	1.3.9	true
pipeline-model-extensions	1.3.9	true
pipeline-rest-api	2.11	true
pipeline-stage-step	2.3	true
pipeline-stage-tags-metadata	1.3.9	true
pipeline-stage-view	2.11	true
plain-credentials	1.5	true
publish-over	0.22	true
publish-over-ssh	1.20.1	true
resource-disposer	0.13	true
scm-api	2.6.3	true
script-security	1.61	true
slave-setup	1.10	true
ssh-credentials	1.17.1	true
ssh-slaves	1.30.1	true
structs	1.19	true
token-macro	2.8	true
warnings-ng	5.3.0	true
windows-slaves	1.4	true
workflow-aggregator	2.6	true
workflow-api	2.35	true
workflow-basic-steps	2.18	true
workflow-cps	2.72	true
workflow-cps-global-lib	2.14	true
workflow-durable-task-step	2.32	true
workflow-job	2.33	true
workflow-multibranch	2.21	true
workflow-scm-step	2.9	true
workflow-step-api	2.20	true
workflow-support	3.3	true

[sparshev]

Yeah, you have alot of plugins there. Let me test with the same list.

[sparshev]

Ok, just tested latest jenkins 2.176.2 with minimal number of plugins (git, Pipeline, Pipeline: Shared Groovy Libraries):

ace-editor 1.1
apache-httpcomponents-client-4-api 4.5.5-3.0
authentication-tokens 1.3
bouncycastle-api 2.17
branch-api 2.5.3
cloudbees-folder 6.9
command-launcher 1.3
credentials 2.2.0
credentials-binding 1.19
display-url-api 2.3.1
docker-commons 1.15
docker-workflow 1.18
durable-task 1.30
git 3.10.1
git-client 2.8.0
git-server 1.7
handlebars 1.1.1
jackson2-api 2.9.9.1
jdk-tool 1.3
jquery-detached 1.2.1
jsch 0.1.55
junit 1.28
lockable-resources 2.5
mailer 1.23
matrix-project 1.14
momentjs 1.1.1
pipeline-build-step 2.9
pipeline-graph-analysis 1.10
pipeline-input-step 2.10
pipeline-milestone-step 1.3.1
pipeline-model-api 1.3.9
pipeline-model-declarative-agent 1.1.1
pipeline-model-definition 1.3.9
pipeline-model-extensions 1.3.9
pipeline-rest-api 2.11
pipeline-stage-step 2.3
pipeline-stage-tags-metadata 1.3.9
pipeline-stage-view 2.11
plain-credentials 1.5
scm-api 2.6.3
script-security 1.61
ssh-credentials 1.17.1
structs 1.19
workflow-aggregator 2.6
workflow-api 2.35
workflow-basic-steps 2.18
workflow-cps 2.72
workflow-cps-global-lib 2.14
workflow-durable-task-step 2.32
workflow-job 2.33
workflow-multibranch 2.21
workflow-scm-step 2.9
workflow-step-api 2.20
workflow-support 3.3

And could confirm - looks like a bug in MPL, will try to figure out.

[sparshev]

Looks like the logic was introduced in workflow-cps-2.71

[sparshev]

https://issues.jenkins-ci.org/browse/JENKINS-31314 related to this change

[sparshev]

And the MPL logic producing the issue: https://github.com/griddynamics/mpl/blob/master/src/com/griddynamics/devops/mpl/Helper.groovy#L125 :

  static void runModule(String src, String path, Map vars = [:]) {
    getShell(vars).evaluate(src, path)
  }

[sparshev]

Right now there is 2 workarounds:

• Ignore the message - it's just a notice (annoying, but still), have no affect to the MPL pipeline execution.
• Revert to workflow-cps-2.70

[sparshev]

Created a ticket to reach help from the jenkins developers: https://issues.jenkins-ci.org/browse/JENKINS-58620

[sparshev]

Just requested update from the workflow-cps plugin developers.

[sparshev]

JENKINS-58620 got solution - hopefully will be resolved in the next groovy-cps (v 1.31) and workflow-cps-plugin (v 2.74).

[sparshev]

The main dependency groovy-cps v1.31 was released with the fix (cloudbees/groovy-cps#101) - so soon the plugin will be updated and the annoying message will never come back.

[sdkrach]

Awesome work! Thank you. We are eagerly awaiting the release.

[sparshev]

Ok, the logic was merged with master - awaiting for workflow-cps-plugin v2.75 release.

[sparshev]

The 2.75 version of workflow-cps-plugin was released - let's check what nightly build will show us...

[sparshev]

I just compared the latest build logs with the previous one - and see that the issue was fixed:

• Previous one TEST-MPLPipelineTest.xml of https://circleci.com/gh/griddynamics/mpl/185#artifacts/containers/0 contains 7 "expected to call" warnings
• Last one TEST-MPLPipelineTest.xml of https://circleci.com/gh/griddynamics/mpl/187#artifacts/containers/0 contains 0 "expected to call" warnings

So the issue was fixed and I'm closing the ticket.

[sdkrach]

Just to confirm, our issues disappeared as well after the update to the most recent version. Thank you for you work!

[sparshev]

@sdkrach great! Thank you for the update)