4.12 cluster with no way to update due to OCPBUGS-13188

[nate-duke]

Describe the bug Have a 4.12.0-0.okd-2023-04-16-041331 cluster with

❯ oc get -o json clusteroperator cloud-controller-manager | jq '.status.conditions[] | select(.type == "Upgradeable")'
{
  "lastTransitionTime": "2023-09-07T20:14:51Z",
  "reason": "AsExpected",
  "status": "False",
  "type": "Upgradeable"
}

OCPBUGS-13188

I'm guessing that wasn't ever ... uh ... backported upstream so we never got it in okd4. Curious why this wasn't mentioned previously but perhaps we waited for stability in 4.13 for too long. I've looked through the various Jira issues I can see but wasn't able to find a workaround and it seems that 4.12.0-0.okd-2023-04-16-041331 is the last okd-4.12 so kcs#7041078

It is identified as a Bug for which cluster must be upgraded to the latest version of 4.12, if not latest then at-least greater than 4.12.18.

Isn't much help here.

Version

IPI vSphere 4.12.0-0.okd-2023-04-16-041331 How reproducible

1 for 1, don't remember this in our other okd4 cluster (which is already on 4.13)

Log bundle

~Pending.~ Complete but it's >13G so it'll take a while to find somewhere to upload and upload it. My maintenance window just opened for the update to 4.13 and i made this discovery. Logs available on request and i'll start a must-gather now but it'll take a while to complete. Will edit when it's done.

[nate-duke]

So, i found https://github.com/okd-project/okd/discussions/1738 (I'd only searched issues)

Sorry if this ends up getting demoted to a discussion, but I'll chase down the context in that thread now.

Edit: Seems that discussion is a dead end.

[nate-duke]

/close I was able to work around this by deleting the cloud-controller-manager operator and allow the cluster version operator to recreate it.