keikoproj / iam-manager

AWS IAM role management for K8s cluster using kube builder "Operator" framework
Apache License 2.0
43 stars 21 forks source link

Support Kubernetes 1.22+ #100

Open diranged opened 2 years ago

diranged commented 2 years ago

Is this a BUG REPORT or FEATURE REQUEST?:

FEATURE REQUEST

What happened:

We've gotten ourselves into a pickle... we were on Kubernetes (EKS) 1.21, and we have upgraded to 1.22. We are now finding our iam-manager pods failing, likely due to incompatible client/API libraries:

2022-05-12T03:32:09.245Z    ERROR   controller-runtime.webhook.webhooks unable to decode the request    {"webhook": "/validate-iammanager-keikoproj-io-v1alpha1-iamrole", "error": "no kind \"AdmissionReview\" is registered for version \"admission.k8s.io/v1\" in scheme \"pkg/runtime/scheme.go:101\""}
github.com/go-logr/zapr.(*zapLogger).Error
    /go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/webhook/admission.(*Webhook).ServeHTTP
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.2/pkg/webhook/admission/http.go:79
sigs.k8s.io/controller-runtime/pkg/webhook.instrumentedHook.func1
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.2/pkg/webhook/server.go:129
net/http.HandlerFunc.ServeHTTP
    /usr/local/go/src/net/http/server.go:2036
net/http.(*ServeMux).ServeHTTP
    /usr/local/go/src/net/http/server.go:2416
net/http.serverHandler.ServeHTTP
    /usr/local/go/src/net/http/server.go:2831
net/http.(*conn).serve
    /usr/local/go/src/net/http/server.go:1919
2022-05-12T03:32:09.245Z    DEBUG   controller-runtime.webhook.webhooks wrote response  {"webhook": "/validate-iammanager-keikoproj-io-v1alpha1-iamrole", "UID": "", "allowed": false, "result": {}, "resultError": "got runtime.Object without object metadata: &Status{ListMeta:ListMeta{SelfLink:,ResourceVersion:,Continue:,RemainingItemCount:nil,},Status:,Message:no kind \"AdmissionReview\" is registered for version \"admission.k8s.io/v1\" in scheme \"pkg/runtime/scheme.go:101\",Reason:,Details:nil,Code:400,}"}

What you expected to happen:

Well ... I had hoped it would work. :)

How to reproduce it (as minimally and precisely as possible):

Try to run the iam-manager on a Kubernetes 1.22+ cluster.

diranged commented 2 years ago

Ok. I take this back ... the issue was that in our effort to migrate to K8S 1.22, we set the admissionReviewVersions field in the webhook to [v1, v1beta1 ]... and the current controller doesn't support v1. Setting it to [v1beta1] works.

That said, most of the libraries in iam-manager are pretty darn old now. On our own custom branch, I was able to update a bunch of them to make them more modern ... but I don't want to continue down that path because our code is forked off of this in a bad way at this point.

figo commented 2 years ago

@diranged do you mind contributing the dependency upgrade back to master branch?

anusha-ragunathan commented 1 year ago

I ran into this issue while running on a Kubernetes 1.25 cluster. The CRDs in the hack folder need to be updated to support the new CRD API changes referenced in https://kubernetes.io/docs/reference/using-api/deprecation-guide/#customresourcedefinition-v122

AnthonyWC commented 1 year ago

I got this error from while updating crd (hack/iam-manager.yaml) for v1.22

2023-02-26T03:18:06.489Z    INFO    internal.config.properties.RunConfigMapInformer Starting config map informer
2023-02-26T03:18:06.893Z    INFO    controller-runtime.metrics  metrics server is starting to listen    {"addr": ":8080"}
2023-02-26T03:18:06.893Z    DEBUG   main.setup  Setting up reconciler with manager
2023-02-26T03:18:06.893Z    INFO    main.setup  region  {"region": "us-west-2"}
2023-02-26T03:18:06.899Z    DEBUG   k8s.client.SetUpEventHandler    Successfully added event broadcaster
2023-02-26T03:18:06.899Z    INFO    v1alpha1.NewWClient loading k8s client
2023-02-26T03:18:06.900Z    INFO    main.setup  Registering controller
I0226 03:18:06.906530       1 leaderelection.go:242] attempting to acquire leader lease  iam-manager-system/controller-leader-election-helper...
2023-02-26T03:18:06.906Z    INFO    controller-runtime.manager  starting metrics server {"path": "/metrics"}
I0226 03:18:24.541801       1 leaderelection.go:252] successfully acquired lease iam-manager-system/controller-leader-election-helper
2023-02-26T03:18:24.541Z    DEBUG   controller-runtime.manager.events   Normal  {"object": {"kind":"ConfigMap","namespace":"iam-manager-system","name":"controller-leader-election-helper","uid":"9836acb6-cbbf-4a48-8591-ed0409222c3a","apiVersion":"v1","resourceVersion":"2591113"}, "reason": "LeaderElection", "message": "iam-manager-controller-manager-6b84cf9756-krjqt_1b2ceddb-623c-492a-8eee-b46067aa4363 became leader"}
2023-02-26T03:18:24.542Z    INFO    controller-runtime.controller   Starting EventSource    {"controller": "iamrole", "source": "kind source: /, Kind="}
2023-02-26T03:18:24.944Z    ERROR   controller-runtime.source   if kind is a CRD, it should be installed before calling Start   {"kind": "Iamrole.iammanager.keikoproj.io", "error": "no matches for kind \"Iamrole\" in version \"iammanager.keikoproj.io/v1alpha1\""}
github.com/go-logr/zapr.(*zapLogger).Error
    /go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.2/pkg/source/source.go:104
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.2/pkg/internal/controller/controller.go:165
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.2/pkg/internal/controller/controller.go:198
sigs.k8s.io/controller-runtime/pkg/manager.(*controllerManager).startLeaderElectionRunnables.func1
    /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.2/pkg/manager/internal.go:473
2023-02-26T03:18:24.945Z    DEBUG   controller-runtime.manager  leader-election runnable finished   {"runnable type": "*controller.Controller"}
2023-02-26T03:18:24.945Z    ERROR   main.setup  problem running manager {"error": "no matches for kind \"Iamrole\" in version \"iammanager.keikoproj.io/v1alpha1\""}
github.com/go-logr/zapr.(*zapLogger).Error
    /go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128
main.main
    /workspace/main.go:109
runtime.main
    /usr/local/go/src/runtime/proc.go:255

apiextensions.k8s.io/v1 depreciated spec.verison and it'd only accept v1. So does image code itself needs updating as i dont have any reference of v1alpha1 in iam-manager.yaml (i only see reference of it in iam-manager_with_webhook.yaml)