Closed camilamacedo86 closed 1 year ago
camilamacedo86
commented
3 years ago Hi, would we have any update on this?
See that we are very close to the release data and fix the projects seems not very hard. See how to fix it in the first comment. Then, would be great to be able to check a new version of your project distributed which is compatible with 4.9.
qrnik
commented
2 years ago Bumping this up.
mtaylor98
commented
2 years ago Also bumping this - I have updated the CRDs and have been able to get them to apply with no issues, but the deployment fails to create the spinnaker-operator container. The halyard container comes up with no issue. The spinnaker-operator container gets the following error:
spinnaker-operator {"level":"info","ts":1645629865.353472,"logger":"cmd","msg":"starting webhook server..."} │
│ spinnaker-operator {"level":"error","ts":1645629865.689998,"logger":"cmd","msg":"error starting webhook server","error":"the server could not find the requested resource","stacktrace":"g │
│ ithub.com/go-logr/zapr.(*zapLogger).Error\n\t/opt/spinnaker-operator/build/vendor/github.com/go-logr/zapr/zapr.go:128\ngithub.com/armory/spinnaker-operator/pkg/operator.Start\n\t/opt/spi │
│ nnaker-operator/build/pkg/operator/operator.go:138\nmain.main\n\t/opt/spinnaker-operator/build/cmd/manager/main.go:24\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:203"}```
ichi0915
commented
2 years ago Hi, I have also updated the CRD and i'm getting the same error as mtaylor98, I think it is related to the go client the operator uses.
I'll trying to update the dependencies to see if that solves the issue.
CB-GuangyaoXie
commented
2 years ago Bumping this up.
ArtemChekunov
commented
2 years ago Bumping this up.
CB-GuangyaoXie
commented
2 years ago Hi, I have also updated the CRD and i'm getting the same error as mtaylor98, I think it is related to the go client the operator uses.
I'll trying to update the dependencies to see if that solves the issue.
@ichi0915 Hi, any luck with updating the dependencies?
ichi0915
commented
2 years ago Hi, this is everything I've done so far ichi0915/spinnaker-operator
Basically I updated the CRDs, the dependencies, the code and the dockerfile.
With this changes I'm able to install the operator in k8s v1.22 without any issue, but when I try to apply the kustomize patches the spinnaker-operator is giving me a similar error as This issue in my case the error is the following:
SpinnakerService validation failed:
Unable to read spinnaker version from manifest:
Unable to read property \"version\":
invalid interface found at versionI'm currently debugin that, but I think the cause has something to do with the function that parses the yaml, and also I saw that seanrclayton has different CRD's than me I'll try with his CRD's to see if it works.
ichi0915
commented
2 years ago Hi, yesterday I did some more changes to de CRD's, I added the flags x-kubernetes-preserve-unknown-fields: true to some fields and now when I apply the kustomize patches the operator parser saves those values and does the deployment succesfully, here is a PR with all the changes, I'll still need do some test to verify everything is fine.
ArtemChekunov
commented
2 years ago Hi @ichi0915,
how is your test?
ichi0915
commented
2 years ago Hi, I ran into some issues but mostly I was having trouble with the openapi command so I decided to try a different migration approach, and also I forgot to upgrade the sdk version. I'm currently working on the upgrade of the sdk and then continue with the other approach.
andrijaperovic
commented
2 years ago Any idea on which version of the Spinnaker Operator v1.22 will be supported? Or is the recommended approach still to update the CRDs, the dependencies, the code and the Dockerfile as @ichi0915 is suggesting?
gnanderson
commented
2 years ago @andrijaperovic We kicked the tyres on this internally and decided we had no bandwidth to help maintain this operator, mostly because it looks like it's not been updated to recent Operator SDK versions in quite a while and there's a lot of work to bring it up to date.
For us at least, this operator was probably the only thing keeping Spinnaker in use on our clusters and our team are now rolling out and migrating to an alternative.
mtaylor98
commented
2 years ago I have started using the updated Halyard and Operator images as well as the updated CRDs which does allow me to deploy the operator and such properly. However, I am now running into an issue where secrets do not work in the SpinnakerService yaml at all. Whether they be specified in the profiles: section or main config section, if they take the format in the documentation as:
encryptedFile:k8s! or as encrypted:k8s! I get an error when the services deploy that say:
│ 2022-07-11 14:38:22.165 ERROR 1 --- [ main] o.s.b.d.LoggingFailureAnalysisReporter : │
│ │
│ *************************** │
│ APPLICATION FAILED TO START │
│ *************************** │
│ │
│ Description: │
│ │
│ Failed to bind properties under 'sql.connectionpools.default.password' to java.lang.String: │
│ │
│ Reason: Secret Engine does not exist: k8s │
│ │
│ Action: │
│ │
│ Update your application's configuration```
rscottwatson
commented
2 years ago @mtaylor98 we had this error too and we were told that we needed to use S3 as a secret store as k8s was not supported in the oss version of the operator despite this https://github.com/armory/spinnaker-operator/blob/master/doc/managing-spinnaker.md#secrets-in-kubernetes-secrets suggesting that k8s was a valid secret engine.
andrijaperovic
commented
2 years ago I have started using the updated Halyard and Operator images as well as the updated CRDs which does allow me to deploy the operator and such properly. However, I am now running into an issue where secrets do not work in the SpinnakerService yaml at all. Whether they be specified in the profiles: section or main config section, if they take the format in the documentation as:
encryptedFile:k8s!or asencrypted:k8s!I get an error when the services deploy that say:│ 2022-07-11 14:38:22.165 ERROR 1 --- [ main] o.s.b.d.LoggingFailureAnalysisReporter : │ │ │ │ *************************** │ │ APPLICATION FAILED TO START │ │ *************************** │ │ │ │ Description: │ │ │ │ Failed to bind properties under 'sql.connectionpools.default.password' to java.lang.String: │ │ │ │ Reason: Secret Engine does not exist: k8s │ │ │ │ Action: │ │ │ │ Update your application's configuration```
@mtaylor98 this is the expected behavior I believe - k8s secret engine is only available with armory operator (https://github.com/armory-io/spinnaker-operator), not the OSS operator & spinnaker.
The only supported secret engines for OSS are GCS/S3: https://spinnaker.io/docs/reference/halyard/secrets/#supported-secret-engines
kalyano
commented
2 years ago @mtaylor98 we had this error too and we were told that we needed to use S3 as a secret store as k8s was not supported in the oss version of the operator despite this https://github.com/armory/spinnaker-operator/blob/master/doc/managing-spinnaker.md#secrets-in-kubernetes-secrets suggesting that k8s was a valid secret engine.
Hi Scott, Can I ask who had recommended you to use S3 vs K8s secret engine? Was this someone from the armory team? I need to dig into why we mentioned K8s secrets on the link you posted; and if required, we will need to change/remove K8s secrets from here.
mtaylor98
commented
2 years ago @kalyano @rscottwatson @andrijaperovic
I am surprised to hear this. Prior to Kubernetes v1.22 I was using Kubernetes Secrets with the OSS operator and Spinnaker with no issue at all. It is only since the updates that this has been unsupported for me and that I have run into any issues. I was using my identical SpinnakerService yaml file and everything from prior to the changes in the upgrades. The docs even point out that Kubernetes Secrets is supported:
https://github.com/armory/spinnaker-operator#benefits-of-operator
Store Spinnaker secrets in Kubernetes secrets.
carlos4ndre
commented
1 year ago any updates on this? estimates to when a fix will be released with an upgrade path to follow?
sergio-quintero
commented
1 year ago Hi all - Latest Operator 1.3.0 release has added support for K8s 1.22+.
Based on the comment. It should be here: https://github.com/spinnaker/spinnaker/issues/6520#issuecomment-910729798
Problem Description
Kubernetes has been deprecating API(s), which will be removed and are no longer available in 1.22. Operators projects using these APIs versions will not work on Kubernetes 1.22 or any cluster vendor using this Kubernetes version(1.22), such as OpenShift 4.9+. Following the APIs that are most likely your projects to be affected by:
Therefore, looks like this project distributes solutions in the repository and does not contain any version compatible with k8s 1.22/OCP 4.9. (More info). Following some findings by checking the distributions published:
NOTE: The above findings are only about the manifests shipped inside of the distribution. It is not checking the codebase.
How to solve
It would be very nice to see new distributions of this project that are no longer using these APIs and so they can work on Kubernetes 1.22 and newer and published in the community-operators collection. OpenShift 4.9, for example, will not ship operators anymore that do still use v1beta1 extension APIs.
Due to the number of options available to build Operators, it is hard to provide direct guidance on updating your operator to support Kubernetes 1.22. Recent versions of the OperatorSDK greater than 1.0.0 and Kubebuilder greater than 3.0.0 scaffold your project with the latest versions of these APIs (all that is generated by tools only). See the guides to upgrade your projects with OperatorSDK Golang, Ansible, Helm or the Kubebuilder one. For APIs other than the ones mentioned above, you will have to check your code for usage of removed API versions and upgrade to newer APIs. The details of this depend on your codebase.
If this projects only need to migrate the API for CRDs and it was built with OperatorSDK versions lower than 1.0.0 then, you maybe able to solve it with an OperatorSDK version >= v0.18.x < 1.0.0:
Alternatively, you can try to upgrade your manifests with controller-gen (version >= v0.4.1) :
If this project does not use Webhooks:
If this project is using Webhooks:
Add the markers sideEffects and admissionReviewVersions to your webhook (Example with sideEffects=None and admissionReviewVersions={v1,v1beta1}: memcached-operator/api/v1alpha1/memcached_webhook.go):
Run the command:
For further information and tips see the comment.