search

jenkinsci / bitbucket-branch-source-plugin

Bitbucket Branch Source Plugin
https://plugins.jenkins.io/cloudbees-bitbucket-branch-source
MIT License
216 stars 351 forks source link

Authentication failure when downloading avatar.png #321

Open maikelvdh opened 4 years ago

maikelvdh commented 4 years ago

Description

We are seeing intermittent failures when avatar.png is attempted to be obtained from BitBucket server when using combination of <username>:<personal access token> as credential set. Why intermittent? As we see that it is happens not consistently while we are not changing the credentials for the configured projects.

2020-06-21 09:13:35.313+0000 [id=657237]    INFO    c.c.j.p.b.BitbucketTeamMetadataAction$BitbucketAvatarCacheSource#fetch: IOException: I/O error when accessing URL: /rest/api/1.0/projects/ACME/avatar.png
com.cloudbees.jenkins.plugins.bitbucket.api.BitbucketRequestException: HTTP request error. Status: 401: .
HttpResponseProxy{HTTP/1.1 401  [X-AREQUESTID: *1M4EBR5x553x10156594x4, X-ASEN: SEN-8840825, WWW-Authenticate: Basic realm="Atlassian Bitbucket", Content-Type: application/json;charset=UTF-8, Date: Sun, 21 Jun 2020 09:13:30 GMT, Location: , Vary: Accept-Encoding, Connection: Keep-Alive] org.apache.http.client.entity.DecompressingEntity@6f558806}
at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getImageRequest(BitbucketServerAPIClient.java:880)
at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getTeamAvatar(BitbucketServerAPIClient.java:731)
Caused: java.io.IOException: I/O error when accessing URL: /rest/api/1.0/projects/ACME/avatar.png
at com.cloudbees.jenkins.plugins.bitbucket.server.client.BitbucketServerAPIClient.getTeamAvatar(BitbucketServerAPIClient.java:736)
at com.cloudbees.jenkins.plugins.bitbucket.BitbucketTeamMetadataAction$BitbucketAvatarCacheSource.fetch(BitbucketTeamMetadataAction.java:80)
at com.cloudbees.jenkins.plugins.bitbucket.avatars.AvatarCache$FetchImage.call(AvatarCache.java:624)
at com.cloudbees.jenkins.plugins.bitbucket.avatars.AvatarCache$FetchImage.call(AvatarCache.java:612)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Similar issues seems to be raised here: https://issues.jenkins-ci.org/browse/JENKINS-61200

Cyanoth commented 4 years ago

We have recently been impacted by this issue after changing the password to a number of accounts; Since the avatar is refreshed periodically & each time it is using the wrong credentials, it eventually it causes a captcha.

Taking a look, when credentials are used on a Bitbucket Team/Project item, the BitbucketAvatarCacheSource configuration gets serialized including the credential it used & stored in JENKINS_HOME/jobs/JOB_NAME/state.xml. This includes the ID & the value of the credential.

So when the credential is updated in the credential store, the BitbucketAvatarCacheSource continues to use the old value from the serialized copy (again, from JENKINS_HOME/jobs/JOB_NAME/state.xml). So as the credential in state.xml is out-of-date, requests to fetch the avatar fail with 401 - Invalid Credentials and serveral of these in a row, cause a captcha.

As a workaround: Create a new credential with valid a username & password/token. It must have a different ID to the originally selected ID. On the bitbucket team/project, change the credential to the new credential. This will refresh the state.xml with the new value of the credential. Optionally, you can change switch back to the old credential id now & delete the newly created one.

Importantly:

This will fix it until the credential is updated again.

I've been looking into a more permanent solution; I believe BitbucketAvatarCacheSource shouldn't use a serialized copy of the credential, instead store the ID of the selected credential & fetch it from the credential store. This is what I've been looking at this afternoon and got a solution working if the credentials are scoped globally. But not when credentials are in a different scope (like the team folder) - and this is likely why the credential has been serialized in the first-place.

Hoping to have a further look into the issue & getting it to work with any scope credentials.

pniederlag commented 3 years ago

@Cyanoth big thx for your detailled analysis and efforts to catch this little bummer.

pniederlag commented 3 years ago

In PR #334 is an attempt to fix the problem.

bitwiseman commented 3 years ago

@pniederlag @Cyanoth Please try the incremental build of the plugin from that PR to see if it addresses this issue.

pniederlag commented 3 years ago

@bitwiseman unfortunatly I can't reliable reproduce the problem yet, it is only happening after some time of jenkins operation. So I can't really test your PR yet.

For us the problem started with an update to bitbucket-branch-source-plugin from 2.7.0 to 2.9.1 while we did NOT change the password of the corresponding user

I have now duplicated the credential to a new credential-id, changed the jobs to use the new credential, and changed back to the original-credential again afterwards. Currently the avatar is displaying correctly. I have added some loggers in order to hopefully understand when and where it is breaking again.

dene14 commented 3 years ago

@bitwiseman Any update on this? or at least any workaround? Plugin version 2.9.2 still have the issue.

plindf commented 3 years ago

@bitwiseman Any update on this? or at least any workaround? Plugin version 2.9.2 still have the issue.

Same over here.

joulrone commented 3 years ago

@bitwiseman Any update on this? or at least any workaround? Plugin version 2.9.2 still have the issue.

I'm still having issues with this.

bitwiseman commented 3 years ago

@joulrone @hasterhorb @dene14 Have you tried the incremental mentioned in https://github.com/jenkinsci/bitbucket-branch-source-plugin/pull/334 ?

justinjschaub commented 3 years ago

Thank you much for the in-depth analysis!

bitwiseman commented 3 years ago

I notice no one responded.

@joulrone @hasterhorb @dene14 @kl1ster @pniederlag Have you tried the incremental mentioned in #334 ?

Fasten90 commented 2 years ago

Existing issue in the 2.9.11 version also.

[Cyanoth] workaround is working: https://github.com/jenkinsci/bitbucket-branch-source-plugin/issues/321#issuecomment-659581948

As a workaround: Create a new credential with valid a username & password/token. It must have a different ID to the originally selected ID. On the bitbucket team/project, change the credential to the new credential. This will refresh the state.xml with the new value of the credential. Optionally, you can change switch back to the old credential id now & delete the newly created one.

Importantly:

* The new credential ID must be different for it to refresh the value stored in state.xml; Simply re-saving or changing credential from one to another doesn't work.

* The new credential must be valid. state.xml only gets update if the new selected credential is valid.
justinjschaub commented 2 years ago

This is still a constant pain. Would love to see the proposed change get merged in.

exoemay commented 1 year ago

Is there a way to disable the avatar pulling completely as a workaround? Even with the proposed workaround above, this is happening at least weekly, sometimes even daily for us anyway and there seems to be no way to stop it.