search

jenkinsci / gitlab-plugin

A Jenkins plugin for interfacing with GitLab
https://plugins.jenkins.io/gitlab-plugin/
GNU General Public License v2.0
1.44k stars 618 forks source link

Very Long gitlabMergeRequestDescription environment variable breaks builds #1724

Open Reamer opened 2 days ago

Reamer commented 2 days ago

Jenkins and plugins versions report

Environment ```text Jenkins: 2.479.1 OS: Linux - 5.15.0-125-generic Java: 21.0.5 - Ubuntu (OpenJDK 64-Bit Server VM) --- antisamy-markup-formatter:162.v0e6ec0fcfcf6 apache-httpcomponents-client-4-api:4.5.14-208.v438351942757 apache-httpcomponents-client-5-api:5.4-124.v31e2987e48f4 asm-api:9.7.1-97.v4cc844130d97 authentication-tokens:1.119.v50285141b_7e1 authorize-project:1.8.1 blueocean-commons:1.27.16 blueocean-rest:1.27.16 bootstrap5-api:5.3.3-1 bouncycastle-api:2.30.1.78.1-248.ve27176eb_46cb_ branch-api:2.1197.vfa_d0c47c267d build-monitor-plugin:1.14-925.v95b_9089a_4c7f build-timeout:1.33 caffeine-api:3.1.8-133.v17b_1ff2e0599 checks-api:2.2.1 cloud-stats:336.v788e4055508b_ cloudbees-folder:6.955.v81e2a_35c08d3 command-launcher:115.vd8b_301cc15d0 commons-compress-api:1.26.1-2 commons-lang3-api:3.17.0-84.vb_b_938040b_078 commons-text-api:1.12.0-129.v99a_50df237f7 conditional-buildstep:1.4.3 config-file-provider:980.v88956a_a_5d6a_d continuum-avm:1.10 cors-filter:1.1 credentials:1389.vd7a_b_f5fa_50a_2 credentials-binding:681.vf91669a_32e45 data-tables-api:2.1.8-1 display-url-api:2.209.v582ed814ff2f docker-commons:443.v921729d5611d docker-java-api:3.4.0-94.v65ced49b_a_7d5 docker-plugin:1.7.0 docker-workflow:580.vc0c340686b_54 durable-task:577.v2a_8a_4b_7c0247 echarts-api:5.5.1-4 eddsa-api:0.3.0-4.v84c6f0f4969e extensible-choice-parameter:1.8.1 external-monitor-job:215.v2e88e894db_f8 font-awesome-api:6.6.0-2 git:5.6.0 git-client:6.1.0 git-parameter:0.10.0 git-server:126.v0d945d8d2b_39 gitlab-api:5.6.0-97.v6603a_83f8690 gitlab-branch-source:711.v41fb_c8b_2b_7a_e gitlab-plugin:1.9.7-SNAPSHOT (private-422a2a46-pdallig) google-oauth-plugin:1.330.vf5e86021cb_ec gson-api:2.11.0-85.v1f4e87273c33 handy-uri-templates-2-api:2.1.8-30.v7e777411b_148 hashicorp-vault-plugin:371.v884a_4dd60fb_6 htmlpublisher:1.36 instance-identity:201.vd2a_b_5a_468a_a_6 ionicons-api:74.v93d5eb_813d5f jackson2-api:2.17.0-379.v02de8ec9f64c jakarta-activation-api:2.1.3-1 jakarta-mail-api:2.1.3-1 javadoc:280.v050b_5c849f69 javax-activation-api:1.2.0-7 javax-mail-api:1.6.2-10 jaxb:2.3.9-1 jdk-tool:80.v8a_dee33ed6f0 jersey2-api:2.44-151.v6df377fff741 jnr-posix-api:3.1.19-2 job-dsl:1.89 joda-time-api:2.13.0-93.v9934da_29b_a_e9 jquery:1.12.4-3 jquery3-api:3.7.1-2 jsch:0.2.16-86.v42e010d9484b_ json-api:20240303-101.v7a_8666713110 json-path-api:2.9.0-118.v7f23ed82a_8b_8 junit:1307.vdd5b_2646279e kubernetes:4295.v7fa_01b_309c95 kubernetes-client-api:6.10.0-240.v57880ce8b_0b_2 kubernetes-credentials:190.v03c305394deb_ ldap:725.v3cb_b_711b_1a_ef lockable-resources:1327.ved786b_a_197e0 m2release:0.16.4 mailer:488.v0c9639c1a_eb_3 matrix-auth:3.2.3 matrix-project:840.v812f627cb_578 maven-artifact-choicelistprovider:1.17 maven-plugin:3.24 metrics:4.2.21-458.vcf496cb_839e4 mina-sshd-api-common:2.14.0-133.vcc091215a_358 mina-sshd-api-core:2.14.0-133.vcc091215a_358 mission-control-view:0.9.16 oauth-credentials:0.653.v14cf2088e950 oic-auth:4.418.vccc7061f5b_6d okhttp-api:4.11.0-172.vda_da_1feeb_c6e openshift-client:1.1.0.424.v829cb_ccf8798 pam-auth:1.11 parameterized-trigger:806.vf6fff3e28c3e pipeline-aggregator-view:104.v94a_e5f6cdb_c3 pipeline-build-step:540.vb_e8849e1a_b_d8 pipeline-graph-analysis:216.vfd8b_ece330ca_ pipeline-groovy-lib:744.v5b_556ee7c253 pipeline-input-step:495.ve9c153f6067b_ pipeline-milestone-step:119.vdfdc43fc3b_9a_ pipeline-model-api:2.2214.vb_b_34b_2ea_9b_83 pipeline-model-definition:2.2214.vb_b_34b_2ea_9b_83 pipeline-model-extensions:2.2214.vb_b_34b_2ea_9b_83 pipeline-rest-api:2.34 pipeline-stage-step:312.v8cd10304c27a_ pipeline-stage-tags-metadata:2.2214.vb_b_34b_2ea_9b_83 pipeline-stage-view:2.34 pipeline-utility-steps:2.18.0 plain-credentials:183.va_de8f1dd5a_2b_ plugin-util-api:5.1.0 run-condition:1.7 scm-api:698.v8e3b_c788f0a_6 script-security:1367.vdf2fc45f229c snakeyaml-api:2.3-123.v13484c65210a_ sonar:2.17.2 ssh-agent:376.v8933585c69d3 ssh-credentials:343.v884f71d78167 ssh-slaves:2.973.v0fa_8c0dea_f9f sshd:3.330.vc866a_8389b_58 structs:338.v848422169819 token-macro:400.v35420b_922dcb_ trilead-api:2.147.vb_73cc728a_32e variant:60.v7290fc0eb_b_cd workflow-aggregator:600.vb_57cdd26fdd7 workflow-api:1336.vee415d95c521 workflow-basic-steps:1058.vcb_fc1e3a_21a_9 workflow-cps:3990.vd281dd77a_388 workflow-durable-task-step:1371.vb_7cec8f3b_95e workflow-job:1468.vcf4f5ee92395 workflow-multibranch:795.ve0cb_1f45ca_9a_ workflow-scm-step:427.v4ca_6512e7df1 workflow-step-api:678.v3ee58b_469476 workflow-support:930.vf51d22b_ce488 ```

What Operating System are you using (both controller, and any agents involved in the problem)?

Ubuntu 22.04

Reproduction steps

  1. Create a Merge-Request with a really huge description
  2. Trigger a Jenkins Pipeline-Job via Webhook-Push

Expected Results

The job runs smoothly

Actual Results

git execution fails

Triggered by GitLab Merge Request #21: misc/renovate/version.quarkus => master
Checking out git git@gitlab.mycompany.com:My-Team/misc/ci-reference.git into /avm/data/jenkins/jobs/misc/jobs/ci-reference-master/workspace@script/f2973eb2a4b1f1753a9f45b0d858da66788cb040b63b64c7c628dd40ee584815 to read Jenkinsfile
The recommended git tool is: NONE
using credential jenkins-sv-01
 > git rev-parse --resolve-git-dir /avm/data/jenkins/jobs/misc/jobs/ci-reference-master/workspace@script/f2973eb2a4b1f1753a9f45b0d858da66788cb040b63b64c7c628dd40ee584815/.git # timeout=10
ERROR: Workspace has a .git repository, but it appears to be corrupt.
hudson.plugins.git.GitException: Error performing git command: git rev-parse --resolve-git-dir /avm/data/jenkins/jobs/misc/jobs/ci-reference-master/workspace@script/f2973eb2a4b1f1753a9f45b0d858da66788cb040b63b64c7c628dd40ee584815/.git
    at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2862)
    at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2766)
    at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2761)
    at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:2051)
    at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:2063)
    at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.hasGitRepo(CliGitAPIImpl.java:405)
    at PluginClassLoader for git-client//hudson.plugins.git.GitAPI.hasGitRepo(GitAPI.java:281)
    at PluginClassLoader for git//hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1205)
    at PluginClassLoader for git//hudson.plugins.git.GitSCM._checkout(GitSCM.java:1310)
    at PluginClassLoader for git//hudson.plugins.git.GitSCM.checkout(GitSCM.java:1277)
    at PluginClassLoader for workflow-scm-step//org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:136)
    at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:167)
    at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:73)
    at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:311)
    at hudson.model.ResourceController.execute(ResourceController.java:101)
    at hudson.model.Executor.run(Executor.java:445)
Caused by: java.io.IOException: Cannot run program "git" (in directory "/avm/data/jenkins/jobs/misc/jobs/ci-reference-master/workspace@script/f2973eb2a4b1f1753a9f45b0d858da66788cb040b63b64c7c628dd40ee584815"): error=7, Argument list too long
    at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1170)
    at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1089)
    at hudson.Proc$LocalProc.<init>(Proc.java:252)
    at hudson.Proc$LocalProc.<init>(Proc.java:221)
    at hudson.Launcher$LocalLauncher.launch(Launcher.java:994)
    at hudson.Launcher$ProcStarter.start(Launcher.java:506)
    at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2839)
    ... 15 more
Caused by: java.io.IOException: error=7, Argument list too long
    at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
    at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:295)
    at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:225)
    at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1126)
    ... 21 more
Cloning the remote Git repository
Cloning repository git@gitlab.mycompany.com:My-Team/misc/ci-reference.git
 > git init /avm/data/jenkins/jobs/misc/jobs/ci-reference-master/workspace@script/f2973eb2a4b1f1753a9f45b0d858da66788cb040b63b64c7c628dd40ee584815 # timeout=10
ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Could not init /avm/data/jenkins/jobs/misc/jobs/ci-reference-master/workspace@script/f2973eb2a4b1f1753a9f45b0d858da66788cb040b63b64c7c628dd40ee584815
    at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl$5.execute(CliGitAPIImpl.java:1073)
    at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:819)
    at PluginClassLoader for git//hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1220)
    at PluginClassLoader for git//hudson.plugins.git.GitSCM._checkout(GitSCM.java:1310)
    at PluginClassLoader for git//hudson.plugins.git.GitSCM.checkout(GitSCM.java:1277)
    at PluginClassLoader for workflow-scm-step//org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:136)
    at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:167)
    at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:73)
    at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:311)
    at hudson.model.ResourceController.execute(ResourceController.java:101)
    at hudson.model.Executor.run(Executor.java:445)
Caused by: hudson.plugins.git.GitException: Error performing git command: git init /avm/data/jenkins/jobs/misc/jobs/ci-reference-master/workspace@script/f2973eb2a4b1f1753a9f45b0d858da66788cb040b63b64c7c628dd40ee584815
    at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2862)
    at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2766)
    at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2761)
    at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:2051)
    at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl$5.execute(CliGitAPIImpl.java:1071)
    ... 10 more
Caused by: java.io.IOException: Cannot run program "git" (in directory "/avm/data/jenkins/jobs/misc/jobs/ci-reference-master/workspace@script/f2973eb2a4b1f1753a9f45b0d858da66788cb040b63b64c7c628dd40ee584815"): error=7, Argument list too long
    at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1170)
    at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1089)
    at hudson.Proc$LocalProc.<init>(Proc.java:252)
    at hudson.Proc$LocalProc.<init>(Proc.java:221)
    at hudson.Launcher$LocalLauncher.launch(Launcher.java:994)
    at hudson.Launcher$ProcStarter.start(Launcher.java:506)
    at PluginClassLoader for git-client//org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2839)
    ... 14 more
Caused by: java.io.IOException: error=7, Argument list too long
    at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
    at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:295)
    at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:225)
    at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1126)
    ... 20 more
ERROR: Error cloning remote repo 'origin'
ERROR: Maximum checkout retry attempts reached, aborting
ERROR: Error cloning remote repo 'origin'
ERROR: Maximum checkout retry attempts reached, aborting
Finished: FAILURE

Anything else?

I only found out by luck that it was due to the long description of the merge request. The background is that I use Renovate. Renovate stores the change log of the libraries in the merge request description. Change logs can become very long.

To solve the problem, I have made the following code change. I know it's a little hack. The 1000 has no deeper meaning except to not overload the environment variable. I would prefer to remove the environment variable completely, but in my opinion this should be configurable. Perhaps some users evaluate the variable.

diff --git a/src/main/java/com/dabsquared/gitlabjenkins/cause/CauseData.java b/src/main/java/com/dabsquared/gitlabjenkins/cause/CauseData.java
index eff3950..b04b58b 100644
--- a/src/main/java/com/dabsquared/gitlabjenkins/cause/CauseData.java
+++ b/src/main/java/com/dabsquared/gitlabjenkins/cause/CauseData.java
@@ -174,7 +174,7 @@ public final class CauseData {
         variables.put("gitlabSourceRepoHttpUrl", sourceRepoHttpUrl);
         variables.putIfNotNull("gitlabMergeCommitSha", mergeCommitSha);
         variables.put("gitlabMergeRequestTitle", mergeRequestTitle);
-        variables.put("gitlabMergeRequestDescription", mergeRequestDescription);
+        variables.put("gitlabMergeRequestDescription", mergeRequestDescription.substring(0, Math.min(mergeRequestDescription.length(), 1000)));
         variables.put("gitlabMergeRequestId", mergeRequestId == null ? "" : mergeRequestId.toString());
         variables.put("gitlabMergeRequestIid", mergeRequestIid == null ? "" : mergeRequestIid.toString());
         variables.put(

Are you interested in contributing a fix?

How would you solve the problem? I would be happy to try to submit a pull request.

krisstern commented 2 days ago

Hi @Reamer, thank you for reporting this bug! We would love to have you submit a patch for this via a PR.

Reamer commented 2 days ago

What kind of pull request should I aim for? 1) The little hack to limit the size to 1000 or an other quite small value 2) Drop gitlabMergeRequestDescription from the env variables 3) Drop gitlabMergeRequestDescription from env variables, which a additional plugin configuration