Jenkins is building "most recent commit" and not necessarily the correct branch for a new PR

[derekatkins]

I and another developer have been working in parallel on three different dev branches in Bitbucket (cloud) in the same project (although it appears this issue can be done with one developer in two branches). The issue appears to be that when Jenkins gets notified, it is not necessarily checking out the correct git branch (or commit!) and, indeed, appears to be using the most recent commit and not the desired branch.

As an example, I am working in branches A and B and my co-worker is working in branch C, but we can ignore him because I was able to obtain this issue all on my own:

I push some changes to branch A. Then I push some changes to branch B. Then I make a PR for branch B. Then I make a PR for branch A.

Both of these will trigger a build, but it'll build the same commit (the last commit on branch B), and it wont build branch A (unless I push another commit into branch A).

I have Jenkins (2.319.2) configured with BBPPR (2.8.1). I have the project configured as: Source: GIT Branches: **

[X] Build with BitBucket Push and Pull Request Plugin Bitbucket Cloud Pull Request Created Bitbucket Cloud Pull Request Updated Push Bitbucket Cloud Push Allowed Branches: master

The BBPPR Hook Log for the project reports:

Last BitBucket Push

Started on Mar 8, 2022 11:53:49 AM
Polling SCM changes on built-in
Using strategy: Default
[poll] Last Built Revision: Revision 7390eea02b41edc8101dda7de55458214afcea31 (origin/BAS-129-normalize-ethernet-up-down-logs)
The recommended git tool is: NONE
using credential 1bc4d338-adf7-48e8-bedf-90ef3c15709f
 > git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/BACnet-C/.git # timeout=10
Fetching changes from the remote Git repositories
 > git config remote.origin.url git@bitbucket.org:xxx.git # timeout=10
Fetching upstream changes from git@bitbucket.org:xxx.git
 > git --version # timeout=10
 > git --version # 'git version 2.27.0'
using GIT_SSH to set credentials 
 > git fetch --tags --force --progress -- git@bitbucket.org:xxx.git +refs/heads/*:refs/remotes/origin/* # timeout=10
Polling for changes in
Seen branch in repository origin/BAS-31
Seen branch in repository origin/BAS-31.1
Seen branch in repository origin/BAS-68
Seen branch in repository origin/BAS-69
Seen branch in repository origin/BAS-73
Seen branch in repository origin/BAS-78
Seen branch in repository origin/BAS-79
Seen branch in repository origin/BAS-8
Seen branch in repository origin/BAS-86
Seen branch in repository origin/BAS-89
Seen branch in repository origin/BAS-90
Seen branch in repository origin/BAS-93
Seen branch in repository origin/DOME-3
Seen branch in repository origin/DOME-4
Seen branch in repository origin/HEAD
Seen branch in repository origin/bas186_7
Seen branch in repository origin/bugfix/BAS-31
Seen branch in repository origin/bugfix/BAS-96
Seen 44 remote branches
 > git show-ref --tags -d # timeout=10
Done. Took 0.69 sec

Unfortunately I have BB Webhook events turned off, and I didn't have debugging turned on.. I can do that (in a test project) if you need logs. Please let me know what else you might need from me.

Honestly, I am very surprised to see this behavior. I THOUGHT it was working before, but I guess I always created the PR shortly after pushing to the PR'd branch and before any other commits got pushed into the repo. But it looks like the trigger just causes it to build the most recent commit in the repo, regardless of what branch that is on.

[derekatkins]

Here's another example, which might provide some more clue. I created a PR on branch BAS-189 (with HEAD commit 5ad3d0b), but it built the wrong branch/commit. Specifically, it built revision 7390eea0 instead of building revision 5ad3d0b. You see, revision 7390eea0 was the "most recent commit" in the repo. I suspect the 'originates from another repository' might be the issue, but I don't know what's causing that. When I look at the bitbucket commit history for this repo, looking at all branches, I see both commits 7390eea0 and 5ad3d0b in the list.

Started by user Derek Atkins: Bitbucket PPR: new pull request created
Running as SYSTEM
Building in workspace /var/lib/jenkins/workspace/BACnet-C
The recommended git tool is: NONE
using credential 1bc4d338-adf7-48e8-bedf-90ef3c15709f
 > git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/BACnet-C/.git # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url git@bitbucket.org:xxx.git # timeout=10
Fetching upstream changes from git@bitbucket.org:xxx.git
 > git --version # timeout=10
 > git --version # 'git version 2.27.0'
using GIT_SSH to set credentials 
 > git fetch --tags --force --progress -- git@bitbucket.org:xxx.git +refs/heads/*:refs/remotes/origin/* # timeout=10
skipping resolution of commit 5ad3d0b14005, since it originates from another repository
Seen branch in repository origin/BAS-103-Detached-Head
Seen branch in repository origin/BAS-106-Fix-1
Seen branch in repository origin/BAS-129-normalize-ethernet-up-down-logs
Seen branch in repository origin/BAS-31
Seen branch in repository origin/BAS-31-Niall
Seen branch in repository origin/BAS-31.1
Seen branch in repository origin/BAS-68
Seen branch in repository origin/BAS-69
Seen branch in repository origin/BAS-73
Seen branch in repository origin/BAS-78
Seen branch in repository origin/BAS-79
Seen branch in repository origin/BAS-8
Seen branch in repository origin/BAS-86
Seen branch in repository origin/BAS-89
Seen branch in repository origin/BAS-90
Seen branch in repository origin/BAS-93
Seen branch in repository origin/DOME-3
Seen branch in repository origin/DOME-4
Seen branch in repository origin/HEAD
Seen branch in repository origin/bas186_7
Seen branch in repository origin/bugfix/BAS-31
Seen branch in repository origin/bugfix/BAS-96
Seen 44 remote branches
 > git show-ref --tags -d # timeout=10
Checking out Revision 7390eea02b41edc8101dda7de55458214afcea31 (origin/BAS-129-normalize-ethernet-up-down-logs)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 7390eea02b41edc8101dda7de55458214afcea31 # timeout=10
Commit message: "Add service to monitor ethernet link and log a normalized Up/Down message"
 > git rev-list --no-walk 7390eea02b41edc8101dda7de55458214afcea31 # timeout=10
Sending build status INPROGRESS for commit 7390eea02b41edc8101dda7de55458214afcea31 to BitBucket is done!
No emails were triggered.
New run name is '#97 - origin/BAS-129-normalize-ethernet-up-down-logs - 7390eea0'
[BACnet-C] $ /bin/sh -xe /tmp/jenkins4790877579759734827.sh
+ rm -rf platforms/archive
+ ./git-info.sh git-properties
Waiting for the completion of [DOME Sentry](https://xxx/)
[DOME Sentry #20](https://xxx/20/) started.
[DOME Sentry #20 - origin/BAS-129-normalize-ethernet-up-down-logs - 7390eea0](https://xxx/20/) completed. Result was SUCCESS
New run name is '#97 - origin/BAS-129-normalize-ethernet-up-down-logs - 7390eea0'
Archiving artifacts
Sending build status SUCCESSFUL for commit 7390eea02b41edc8101dda7de55458214afcea31 to BitBucket is done!
No emails were triggered.
Finished: SUCCESS

[derekatkins]

Doing some googling, I found https://github.com/jenkinsci/gitlab-plugin/issues/444 which pointed me to https://github.com/jenkinsci/gitlab-plugin/issues/477 -- so I looked at my Test Repo / Project that I've been using for testing (where the build script in Jenkins runs printenv) and noticed that bitbucket appears to be sending the git URL as https:

BITBUCKET_PAYLOAD={"repository":{"scm":"git","name":"test-repo","links":{"html":{"href":"https://bitbucket.org/xxx/test-repo"}...

whereas I have the git URL set to SSH in Jenkins... So it doesn't think it is the same repo.

I'm going to research if there is a way to get BB to report the URL in ssh format, or some way to tell Jenkins to equate the https URI to an SSH URI. Unless you've seen this before?

[derekatkins]

I asked Atlassian about it (specifically about including the SSH URI in the webhook), and got this reply:

I've reviewed your request and, my understanding is that you are having issues with your Jenkins integration due to the SSH clone URL not being present on your webhook's payload body. Would that be correct?

If so, I'm afraid that this field should not be present in the payload by design. Since webhooks work basically with HTTPS protocol, all URLs are worked out on the HTTPS format and, for the repo push, you should be getting only an URL for the repo as a reference.

One thing that you can do though would be to grab the repo HTTPS URL and use that to perform an API call to our repositories endpoint:

curl -X GET "https://api.bitbucket.org/2.0/repositories/<workspace>/<repository>/" Within the payload of this API call, you should be able to find the SSH URL as shown below:

"clone": [
{
"href": "https://mtizotti@bitbucket.org/mtizotti/2ndrepo.git",
"name": "https"
},
{
"href": "git@bitbucket.org:mtizotti/2ndrepo.git",
"name": "ssh"
}
]

Let me know if this works as a workaround for you and if you have any further questions, I'll be glad to help

Is there any way we could use this in the plug-in to "fix" the skipping resolution of commit 5ad3d0b14005, since it originates from another repository error?

[derekatkins]

See also https://issues.jenkins.io/browse/JENKINS-67073

[derekatkins]

See also https://jira.atlassian.com/browse/BCLOUD-21788 -- a feature request to add an SSH URI to the BB webhook.

[mkyc]

I just had similar problem and I solved it with changing:

        cpsScm {
            scm {
                git {
                    remote {
                        url('git@bitbucket.org:some/repo.git')
                        credentials('some-creds')
                    }
                    branches('')
                }
            }
            scriptPath('some/path.jenkinsfile')
        }

into:

        cps {
            script(readFileFromWorkspace('some/path.jenkinsfile'))
            sandbox()
        }

[derekatkins]

@mkyc -- where did you make that change? That looks like pipeline code? I'm using a freestyle project, so I have no idea how to make such a change.

[mkyc]

@derekatkins you're right. I'm still fighting with this plugin and it looks like I will use freestyle job as triggering and pipeline job as workhorse code being invoked from freestyle one.

[derekatkins]

@mkyc I'm not 100% sure it's a problem with this plugin.. It's possibly a bug in the GIT SCM plugin. Indeed, that appears to be where the error skipping resolution of commit 5ad3d0b14005, since it originates from another repository comes from. But I do not know if there is something that this plugin can do to tell GIT-SCM that the requested commit is the same.

Are you using HTTPS or SSH access to bitbucket from Jenkins?

[TueChristensen]

I just had the same issue as mentioned. Caused us to merge a faulty branch and break our main trunk.

As far as I can see nobody had a solution for this?

[TueChristensen]

Just to re-confirm, but this is still a problem running

Bitbucket Cloud BPPR pluging 3.0.2 Jenkins 2.444

Unfortunately, this makes the plugin almost useless for reporting status build on PRs as it almost always reports back on the wrong commit.

[anassar-lab]

We were able to resolve this via a workaround, we use bitbucket for source control, so we have configured Jenkins git repo to use username/password instead and made sure that the repo name matches what is received over the webhook, for bitbucket the URL format would be https://bitbucket.org/ORG/REPO Note that we needed to remove the .git from the URL repo configuration on Jenkins for this workaround to work.

Jenkins 2.426.3 Bitbucket Cloud Bitbucket Build Status Notifier 1.4.2 Bitbucket Push and Pull Request 3.0.2

[julioc-p]

This problem stems from the git plugin, and the way it selects the commits. A new option will be added to enforce the git plugin to use a specific branch

[julioc-p]

You can use this as well

[derekatkins]

You can use this as well

Thanks @julioc-p . Is this a new variable or will this work with "older" versions of the plugin? (Or, a better question, what minimum version is required to make this work?)

Also, I guess I would need to put this into EACH trigger?

[cdelmonte-zg]

Hi @derekatkins, the envvar BITBUCKET_SOURCE_BRANCH is present since version 1.5. and, if I understand it right, it should be written in the branch filter field of the git plugin, not in the one of the BPPR triggers…