[JENKINS-51179] Pipeline dies with `java.lang.String cannot be cast to hudson.model.Fingerprint`

[timja]

All builds that I checked fail with following stack :

java.lang.ClassCastException: java.lang.String cannot be cast to hudson.model.Fingerprint
 at hudson.model.Fingerprint.load(Fingerprint.java:1369)
 at hudson.model.Fingerprint.load(Fingerprint.java:1357)
 at hudson.model.FingerprintMap.load(FingerprintMap.java:97)
 at hudson.model.FingerprintMap.load(FingerprintMap.java:47)
 at hudson.util.KeyedDataStorage.get(KeyedDataStorage.java:161)
 at hudson.model.FingerprintMap.get(FingerprintMap.java:82)
 at hudson.model.FingerprintMap.get(FingerprintMap.java:47)
 at hudson.util.KeyedDataStorage.getOrCreate(KeyedDataStorage.java:111)
 at hudson.model.FingerprintMap.getOrCreate(FingerprintMap.java:68)
 at hudson.model.FingerprintMap.getOrCreate(FingerprintMap.java:64)
 at com.cloudbees.plugins.credentials.CredentialsProvider.getOrCreateFingerprintOf(CredentialsProvider.java:1403)
 at com.cloudbees.plugins.credentials.CredentialsProvider.trackAll(CredentialsProvider.java:1457)
 at com.cloudbees.plugins.credentials.CredentialsProvider.track(CredentialsProvider.java:1421)
 at hudson.plugins.git.GitSCM.createClient(GitSCM.java:818)
 at hudson.plugins.git.GitSCM.createClient(GitSCM.java:786)
 at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1154)
 at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:113)
 at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:144)
 at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:120)
 at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:298)
 at hudson.model.ResourceController.execute(ResourceController.java:97)
 at hudson.model.Executor.run(Executor.java:429)
 Finished: FAILURE

---

Originally reported by jozue_noon, imported from: Pipeline dies with `java.lang.String cannot be cast to hudson.model.Fingerprint`

• assignee: oleg_nenashev
• status: Resolved
• priority: Critical
• resolution: Fixed
• resolved: 2018-06-06T09:47:42+00:00
• imported: 2022/01/10

[timja]

oleg_nenashev:

Likely the Fingerprint file is corrupted (or uses wrong XML notation)
Please provide...

• Version of Jenkins
• Failing fingerprint file if you can determine it (probably not)

The error does not get propagated correctly in the code, and the unchecked cast is a bad idea. It needs some polishing at least.

JENKINS-49588 may be related

[timja]

olira:

Hi Oleg,

we are seeing the same issue at the moment.
Jenkins Version is 2.112

I assume that in our case this has happened after the disk being full and Jenkins getting into an inconsistent state. Jenkins was still working but wasn't accepting webhook builds anymore. After freeing up some space, It started accepting webhooks again but is now failing with the above exception.

Is there any way to get it valid again without restarting Jenkins?

[timja]

oleg_nenashev:

According to the current stacktrace, you "just" need to find corrupted Fingerprint files and delete/move/fix them. The file is not loaded into the memory, so there should be no need in restart once the file is fixed somehow. If you delete it, a new file will be just created. You will lose track of credentials usage, but it should be flawless in other terms

[timja]

olira:

Hi Oleg,

Thanks for the reply. Here is what I've done to find existing files that are corrupted:

grep -rL "" $JENKINS_HOME/fingerprints

I found two files, one of them empty, the other one indeed corrupted (not written to the end), so I deleted both. This didn't solve the issue.

I restarted Jenkins, still no luck.

Then I was digging down into where exactly the issue was occurring. I found that multiple withCredential blocks with different credentials were failing with above exception. As  I didn't know where else the fingerprint could be stored I simply updated that credentials and now builds are working again.

[timja]

oleg_nenashev:

I am still not sure what causes it in your case, but I have created https://github.com/jenkinsci/jenkins/pull/3439 which improves diagnostics and propagates proper errors to the Fingerprint loading engine. At least users will see which file causes issues

[timja]

scm_issue_link:

Code changed in jenkins
User: Oleg Nenashev
Path:
test/src/test/java/hudson/model/FingerprintTest.java
http://jenkins-ci.org/commit/jenkins/cc9425039aa2f8f46153e404c226cf98b0e65659
Log:
JENKINS-51179 - Reproduce the issue in Test

[timja]

scm_issue_link:

Code changed in jenkins
User: Oleg Nenashev
Path:
core/src/main/java/hudson/model/Fingerprint.java
http://jenkins-ci.org/commit/jenkins/922617509eccbcb89ad3bb8b5f6e33bd5a8182c0
Log:
JENKINS-51179 - Check type of the loaded object when loading fingerprints

[timja]

scm_issue_link:

Code changed in jenkins
User: Oleg Nenashev
Path:
core/src/main/java/hudson/model/Fingerprint.java
test/src/test/java/hudson/model/FingerprintTest.java
http://jenkins-ci.org/commit/jenkins/9e64bcdcb4a2cf12d59dfa334e09ffb448d361e9
Log:
Merge pull request #3439 from oleg-nenashev/bug/JENKINS-51179

JENKINS-51179 - Provide diagnostics when loading corrupted fingerprint files

Compare: https://github.com/jenkinsci/jenkins/compare/cdda62c35a86...9e64bcdcb4a2
*NOTE:* This service been marked for deprecation: https://developer.github.com/changes/2018-04-25-github-services-deprecation/

Functionality will be removed from GitHub.com on January 31st, 2019.

[timja]

oleg_nenashev:

Fixed in 2.123

[timja]

jozue_noon:

Thank you!

[timja]

[Originally duplicated by: JENKINS-23566]