CustomResourceDefinition.apiextensions.k8s.io "datadogagents.datadoghq.com" is invalid: status.storedVersions[0]: Invalid value: "v1alpha1": must appear in spec.versions

[diranged]

Describe what happened:

CustomResourceDefinition.apiextensions.k8s.io "datadogagents.datadoghq.com" is invalid: status.storedVersions[0]: Invalid value: "v1alpha1": must appear in spec.versions

We receive the above error when we tried to upgrade the datadog-operator chart from 1.8.6 to 2.0.0... I cannot see a clean path forward to the upgrade. What are we missing?

[aquiladayc]

Hello @diranged Couldn't replicate the same error with what you provided but as it mentions v1alpha1, possibly migration was not properly done in your environment. Please see: https://docs.datadoghq.com/containers/guide/datadogoperator_migration/#migrating-to-version-10-of-the-datadog-operator

Or, if old installation is making the issue, you can simply uninstall existing resources and clean install with the latest version. (Please make sure your repo is up to date with helm repo update)

If you need more assistance, please feel free to reach our support https://www.datadoghq.com/support/ To proceed smoothly, we would like to know:

• Steps to replicate (commands you run)
• Your config files Thank you

[diranged]

So - we ran this upgrade across 15 clusters... and 10 of them failed with the prioblem describd above, and 5 worked... no idea why. Deleting the DatadogAgent resource wasn't an option for us (though I did try it on a test cluster)... but I found this thread: https://stackoverflow.com/questions/59452368/how-to-remove-version-from-status-storedversions-while-manually-upgrading-the-ex

The following commands worked to resolve the issue:

% kubectl proxy &
% curl -d '[{ "op": "replace", "path":"/status/storedVersions", "value": ["v2alpha1"] }]' \
  -H "Content-Type: application/json-patch+json" \
  -X PATCH localhost:8001/apis/apiextensions.k8s.io/v1/customresourcedefinitions/datadogagents.datadoghq.com/status