search

jenkinsci / gitlab-branch-source-plugin

A Jenkins Plugin for GitLab Multibranch Pipeline jobs and Folder Organization
https://plugins.jenkins.io/gitlab-branch-source
MIT License
121 stars 85 forks source link

Scan Gitlab Group failed if there's a open MR from a private fork repository #306

Open X1aomu opened 1 year ago

X1aomu commented 1 year ago

Jenkins and plugins versions report

Environment ```text Jenkins: 2.400 OS: Windows Server 2016 - 10.0 Java: 17.0.2 - Oracle Corporation (OpenJDK 64-Bit Server VM) --- analysis-model-api:11.1.0 ant:481.v7b_09e538fcca antisamy-markup-formatter:159.v25b_c67cd35fb_ apache-httpcomponents-client-4-api:4.5.14-150.v7a_b_9d17134a_5 authentication-tokens:1.53.v1c90fd9191a_b_ blueocean:1.27.3 blueocean-autofavorite:1.2.5 blueocean-bitbucket-pipeline:1.27.3 blueocean-commons:1.27.3 blueocean-config:1.27.3 blueocean-core-js:1.27.3 blueocean-dashboard:1.27.3 blueocean-display-url:2.4.1 blueocean-events:1.27.3 blueocean-git-pipeline:1.27.3 blueocean-github-pipeline:1.27.3 blueocean-i18n:1.27.3 blueocean-jwt:1.27.3 blueocean-personalization:1.27.3 blueocean-pipeline-api-impl:1.27.3 blueocean-pipeline-editor:1.27.3 blueocean-pipeline-scm-api:1.27.3 blueocean-rest:1.27.3 blueocean-rest-impl:1.27.3 blueocean-web:1.27.3 bootstrap5-api:5.2.2-2 bouncycastle-api:2.27 branch-api:2.1071.v1a_188a_562481 build-timeout:1.30 caffeine-api:2.9.3-65.v6a_47d0f4d1fe checks-api:2.0.0 cloudbees-bitbucket-branch-source:800.va_b_b_9a_a_5035c1 cloudbees-folder:6.815.v0dd5a_cb_40e0e command-launcher:90.v669d7ccb_7c31 commons-lang3-api:3.12.0-36.vd97de6465d5b_ commons-text-api:1.10.0-36.vc008c8fcda_7b_ configuration-as-code:1625.v27444588cc3d credentials:1224.vc23ca_a_9a_2cb_0 credentials-binding:604.vb_64480b_c56ca_ data-tables-api:1.13.3-3 display-url-api:2.3.7 durable-task:504.vb10d1ae5ba2f echarts-api:5.4.0-3 email-ext:2.96 extended-choice-parameter:359.v35dcfdd0c20d favorite:2.4.1 file-operations:1.11 font-awesome-api:6.3.0-2 forensics-api:2.1.0 git:5.0.0 git-client:4.2.0 git-forensics:2.0.0 github:1.37.0 github-api:1.303-417.ve35d9dd78549 github-branch-source:1703.vd5a_2b_29c6cdc gitlab-api:5.2.0-86.v1ed41a_9cf486 gitlab-branch-source:650.va_d1ce6d01959 gitlab-plugin:1.7.11 gradle:2.4 handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953 htmlpublisher:1.31 instance-identity:142.v04572ca_5b_265 ionicons-api:45.vf54fca_5d2154 jackson2-api:2.14.2-319.v37853346a_229 jakarta-activation-api:2.0.1-3 jakarta-mail-api:2.0.1-3 javax-activation-api:1.2.0-6 javax-mail-api:1.6.2-9 jaxb:2.3.8-1 jdk-tool:63.v62d2fd4b_4793 jenkins-design-language:1.27.3 jersey2-api:2.39.1-1 jjwt-api:0.11.5-77.v646c772fddb_0 job-dsl:1.83 jobConfigHistory:1207.vd28a_54732f92 jquery:1.12.4-1 jquery3-api:3.6.4-1 jsch:0.1.55.61.va_e9ee26616e7 junit:1196.vb_4cf28b_c7724 ldap:671.v2a_9192a_7419d locale:262.ved03281fa_64f localization-support:1.2 localization-zh-cn:1.0.24 mailer:448.v5b_97805e3767 matrix-auth:3.1.7 matrix-project:785.v06b_7f47b_c631 mina-sshd-api-common:2.9.2-62.v199162f0a_2f8 mina-sshd-api-core:2.9.2-62.v199162f0a_2f8 okhttp-api:4.10.0-132.v7a_7b_91cef39c pam-auth:1.10 pipeline-build-step:488.v8993df156e8d pipeline-github-lib:42.v0739460cda_c4 pipeline-graph-analysis:202.va_d268e64deb_3 pipeline-groovy-lib:656.va_a_ceeb_6ffb_f7 pipeline-input-step:466.v6d0a_5df34f81 pipeline-milestone-step:111.v449306f708b_7 pipeline-model-api:2.2125.vddb_a_44a_d605e pipeline-model-definition:2.2125.vddb_a_44a_d605e pipeline-model-extensions:2.2125.vddb_a_44a_d605e pipeline-rest-api:2.32 pipeline-stage-step:305.ve96d0205c1c6 pipeline-stage-tags-metadata:2.2125.vddb_a_44a_d605e pipeline-stage-view:2.32 plain-credentials:143.v1b_df8b_d3b_e48 plugin-util-api:3.2.0 popper2-api:2.11.6-2 powershell:2.0 prism-api:1.29.0-4 pubsub-light:1.17 resource-disposer:0.22 scm-api:631.v9143df5b_e4a_a script-security:1244.ve463715a_f89c snakeyaml-api:1.33-95.va_b_a_e3e47b_fa_4 sse-gateway:1.26 ssh-agent:327.v230ecd01f86f ssh-credentials:305.v8f4381501156 ssh-slaves:2.877.v365f5eb_a_b_eec sshd:3.275.v9e17c10f2571 structs:324.va_f5d6774f3a_d timestamper:1.24 token-macro:321.vd7cc1f2a_52c8 trilead-api:2.84.v72119de229b_7 variant:59.vf075fe829ccb warnings-ng:10.1.0 workflow-aggregator:596.v8c21c963d92d workflow-api:1208.v0cc7c6e0da_9e workflow-basic-steps:1010.vf7a_b_98e847c1 workflow-cps:3653.v07ea_433c90b_4 workflow-durable-task-step:1244.vee71f675dee6 workflow-job:1289.vd1c337fd5354 workflow-multibranch:733.v109046189126 workflow-scm-step:408.v7d5b_135a_b_d49 workflow-step-api:639.v6eca_cd8c04a_a_ workflow-support:839.v35e2736cfd5c ws-cleanup:0.45 ```

What Operating System are you using (both controller, and any agents involved in the problem)?

Windows (should not matter)

Reproduction steps

  1. Create an Organization Folder job Job that make use of the plugin, with the checkout credentials Key.
  2. Set the project owner to a subgroup, named foo/bar
  3. A gitlab project under foo/bar named P
  4. P have an open MR created from a fork someone/P that is a private fork, which gives no access to the checkout credentials Key.
  5. Run the Scan Gitlab Group

Expected Results

During the scanning process of foo/bar/P, the MR is found. The plugin found that it cannot clone the merge-head revision. Thus just ignore this MR, and continue to scan the leaving things.

Actual Results

The entire Scan Gitlab Group failed, 

Checking merge request !MR
ERROR: [周日 4月 23 14:29:44 CST 2023] Could not fetch sources from navigator io.jenkins.plugins.gitlabbranchsource.GitLabSCMNavigator@41831ee7
[周日 4月 23 14:29:44 CST 2023] Finished organization scan. Scan took 6 分 21 秒
FATAL: Failed to recompute children of Job
org.gitlab4j.api.GitLabApiException: 404 Project Not Found
    at org.gitlab4j.api.AbstractApi.validate(AbstractApi.java:678)
    at org.gitlab4j.api.AbstractApi.get(AbstractApi.java:214)
    at org.gitlab4j.api.ProjectApi.getProject(ProjectApi.java:748)
    at org.gitlab4j.api.ProjectApi.getProject(ProjectApi.java:680)
    at io.jenkins.plugins.gitlabbranchsource.GitLabSCMSource.retrieve(GitLabSCMSource.java:420)
Caused: java.io.IOException: Failed to fetch latest heads
    at io.jenkins.plugins.gitlabbranchsource.GitLabSCMSource.retrieve(GitLabSCMSource.java:537)
    at jenkins.scm.api.SCMSource._retrieve(SCMSource.java:373)
    at jenkins.scm.api.SCMSource.fetch(SCMSource.java:327)
    at jenkins.branch.MultiBranchProjectFactory$BySCMSourceCriteria.recognizes(MultiBranchProjectFactory.java:261)
    at jenkins.branch.OrganizationFolder$SCMSourceObserverImpl$1.recognizes(OrganizationFolder.java:1358)
    at jenkins.branch.OrganizationFolder$SCMSourceObserverImpl$1.complete(OrganizationFolder.java:1373)
    at jenkins.scm.api.trait.SCMNavigatorRequest.process(SCMNavigatorRequest.java:254)
    at jenkins.scm.api.trait.SCMNavigatorRequest.process(SCMNavigatorRequest.java:204)
    at io.jenkins.plugins.gitlabbranchsource.GitLabSCMNavigator.visitSources(GitLabSCMNavigator.java:294)
    at jenkins.branch.OrganizationFolder.computeChildren(OrganizationFolder.java:535)
    at com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:269)
    at com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:167)
    at jenkins.branch.OrganizationFolder$OrganizationScan.run(OrganizationFolder.java:917)
    at hudson.model.ResourceController.execute(ResourceController.java:101)
    at hudson.model.Executor.run(Executor.java:442)
Finished: FAILURE

Anything else?

No response

Deklin commented 2 months ago

Any solution to this? I have a workaround where the forked repo must add a user that is based on the credentials configured in the system level jenkins properties for gitlab, but hoping for a better solution

X1aomu commented 1 month ago

Hi @Deklin

A similar workaround here. We met a consensus that all of the forking respositories must share itself to an ancestor group of the original repository, instead of a user. As the group access token is configured as the credentials. That might be easier than user-based approach.

Hope it helps.