search

jenkinsci / bitbucket-branch-source-plugin

Bitbucket Branch Source Plugin
https://plugins.jenkins.io/cloudbees-bitbucket-branch-source
MIT License
216 stars 353 forks source link

SCM fetch fails due to neglected port of bitbucket server #813

Open schorfi opened 7 months ago

schorfi commented 7 months ago

Jenkins and plugins versions report

Environment ```text Jenkins: 2.426.3 OS: Linux - 6.2.0-1018-aws Java: 17.0.9 - Eclipse Adoptium (OpenJDK 64-Bit Server VM) --- ant:497.v94e7d9fffa_b_9 antisamy-markup-formatter:162.v0e6ec0fcfcf6 apache-httpcomponents-client-4-api:4.5.14-208.v438351942757 atlassian-bitbucket-server-integration:4.0.0 authentication-tokens:1.53.v1c90fd9191a_b_ bootstrap5-api:5.3.2-3 bouncycastle-api:2.30.1.77-225.v26ea_c9455fd9 branch-api:2.1148.vce12cfcdf090 build-timeout:1.32 caffeine-api:3.1.8-133.v17b_1ff2e0599 checks-api:2.0.2 cloudbees-bitbucket-branch-source:874.v659a_b_70f5e69 cloudbees-folder:6.858.v898218f3609d commons-lang3-api:3.13.0-62.v7d18e55f51e2 commons-text-api:1.11.0-95.v22a_d30ee5d36 credentials:1319.v7eb_51b_3a_c97b_ credentials-binding:657.v2b_19db_7d6e6d display-url-api:2.200.vb_9327d658781 durable-task:550.v0930093c4b_a_6 echarts-api:5.4.3-2 email-ext:2.104 font-awesome-api:6.5.1-2 git:5.2.1 git-client:4.6.0 github:1.38.0 github-api:1.318-461.v7a_c09c9fa_d63 github-branch-source:1772.va_69eda_d018d4 gradle:2.10 gson-api:2.10.1-15.v0d99f670e0a_7 handy-uri-templates-2-api:2.1.8-30.v7e777411b_148 instance-identity:185.v303dc7c645f9 ionicons-api:56.v1b_1c8c49374e jackson2-api:2.16.1-373.ve709c6871598 jakarta-activation-api:2.0.1-3 jakarta-mail-api:2.0.1-3 javax-activation-api:1.2.0-6 javax-mail-api:1.6.2-9 jaxb:2.3.9-1 jjwt-api:0.11.5-77.v646c772fddb_0 joda-time-api:2.12.7-29.v5a_b_e3a_82269a_ jquery3-api:3.7.1-1 json-path-api:2.9.0-33.v2527142f2e1d junit:1259.v65ffcef24a_88 ldap:711.vb_d1a_491714dc mailer:463.vedf8358e006b_ matrix-auth:3.2.1 matrix-project:822.824.v14451b_c0fd42 mina-sshd-api-common:2.12.0-90.v9f7fb_9fa_3d3b_ mina-sshd-api-core:2.12.0-90.v9f7fb_9fa_3d3b_ okhttp-api:4.11.0-172.vda_da_1feeb_c6e pam-auth:1.10 pipeline-build-step:540.vb_e8849e1a_b_d8 pipeline-github-lib:42.v0739460cda_c4 pipeline-graph-analysis:202.va_d268e64deb_3 pipeline-groovy-lib:704.vc58b_8890a_384 pipeline-input-step:477.v339683a_8d55e pipeline-milestone-step:111.v449306f708b_7 pipeline-model-api:2.2175.v76a_fff0a_2618 pipeline-model-definition:2.2175.v76a_fff0a_2618 pipeline-model-extensions:2.2175.v76a_fff0a_2618 pipeline-rest-api:2.34 pipeline-stage-step:305.ve96d0205c1c6 pipeline-stage-tags-metadata:2.2175.v76a_fff0a_2618 pipeline-stage-view:2.34 plain-credentials:143.v1b_df8b_d3b_e48 plugin-util-api:3.8.0 resource-disposer:0.23 scm-api:683.vb_16722fb_b_80b_ script-security:1326.vdb_c154de8669 snakeyaml-api:2.2-111.vc6598e30cc65 sonar:2.17.1 ssh-credentials:308.ve4497b_ccd8f4 ssh-slaves:2.948.vb_8050d697fec structs:337.v1b_04ea_4df7c8 timestamper:1.26 token-macro:400.v35420b_922dcb_ trilead-api:2.133.vfb_8a_7b_9c5dd1 variant:60.v7290fc0eb_b_cd workflow-aggregator:596.v8c21c963d92d workflow-api:1291.v51fd2a_625da_7 workflow-basic-steps:1042.ve7b_140c4a_e0c workflow-cps:3867.v535458ce43fd workflow-durable-task-step:1331.vc8c2fed35334 workflow-job:1385.vb_58b_86ea_fff1 workflow-multibranch:773.vc4fe1378f1d5 workflow-scm-step:415.v434365564324 workflow-step-api:657.v03b_e8115821b_ workflow-support:865.v43e78cc44e0d ws-cleanup:0.45 ```

What Operating System are you using (both controller, and any agents involved in the problem)?

see above

Reproduction steps

  1. install jenkins and that plugin (my case: docker container)
  2. configure jenkins to use bitbucket endpoint (bitbucket server) with non-default port 1.1 e.g. https://your.bitbucketserver.com:9443/
  3. setup multibranch pipeline 2.1 using bitbucket as your branch source 2.2 select your configured server and credentials 2.3 see success of that connection by finding and selecting the project/owner and the repo-names 2.4 save your changes
  4. trigger build pipeline
  5. observe that each branch SCM checkout

Expected Results

successful branch checkouts in step 4 fetch from same URL as configured in jenkins and used for branch detection

4.1 logs: git config remote.origin.url https://your.bitbucketserver.com:9443/scm/yourproject/yourrepo.git # timeout=10

Actual Results

step

  1. observe that each branch SCM checkout times out 4.1 logs: git config remote.origin.url https://your.bitbucketserver.com/scm/yourproject/yourrepo.git # timeout=10

see missing sample port :9443 above!

Anything else?

please support HTTP access tokens for repositories ;)

Are you interested in contributing a fix?

No response

andrey-fomin commented 7 months ago

For local testing I use Bitbucket on https://bitbucket-primary.local:7990/bitbucket. I don't see any issues with port. Please can you check what clone link is returned by bitbucket API?

https://developer.atlassian.com/server/bitbucket/rest/v818/api-group-project/#api-api-latest-projects-projectkey-repos-repositoryslug-get

For example https://bitbucket-primary.local:7990/bitbucket/rest/api/latest/projects/PROJECT_1/repos/rep_1

{
  ...
  "links": {
    "clone": [
      {
        "href": "ssh://git@bitbucket-primary.local:7999/project_1/rep_1.git",
        "name": "ssh"
      },
      {
        "href": "https://bitbucket-primary.local:7990/bitbucket/scm/project_1/rep_1.git",
        "name": "http"
      }
    ],
  }
  ...
}
schorfi commented 7 months ago

@andrey-fomin thank you for picking this up

I see where you are heading to, instead of using the "Instance URL" or "Server URL" for the git config repo, it falls back to the clone URLs. This becomes interesting now, as the server URL is not directly reachable for the Jenkins server/host. It requires an additional hop to bridge to a different network, where one network's routing is not in my control. This hop is kind of a proxy (e.g. load balancer whatever)

  "links": {
    "clone": [
      {
        "href": "ssh://git@my.bitbucketserver.com:7999/myProject/myRepo.git",
        "name": "ssh"
      },
      {
        "href": "https://my.bitbucketserver.com/bitbucket/scm/myProject/myRepo.git",
        "name": "http"
      }
    ],
  }
}
andrey-fomin commented 7 months ago

Ok. Then it means that clone links should not be used at all. The repository clone link should be constructed by bitbucket address, project name and repo name. I believe the same problem is also applicable to SSH if ssh tunnel is used.

And yes before #796 http clone links from API response was not used. There was a lot of code to handle them but finally http clone link was just constructed by server address, project and repo name.

Then I have question to maintainers. What is the better way to fix this problem. I see following approaches

  1. Don't use http clone link from API for primary bitbucket instance. It can help to handle http reverse proxies. But I don't understand what to do with SSH tunnels in such case. I believe http and ssh protocols should be handled in the same way.
  2. Provide extra configuration parameters for Bitbucket Server in System settings.
    • Generate http clone link using server url - boolean parameter. If set to true then don't use http clone link from API but generate it. Not compatible with mirroring.
    • SSH server url- optional string parameter. If it's not empty then don't use ssh clone link from API but generate it. Not compatible with mirroring.

@lifeofguenter @raul-arabaolaza