search

Kong / kong-operator

Kong Operator for Kubernetes and OpenShift
https://konghq.com
Apache License 2.0
58 stars 27 forks source link

Can't create Kong on Openshift 4.9.0 #71

Closed nautiam closed 2 years ago

nautiam commented 2 years ago

I install Kong-operator on Openshift 4.9.0 from RedHat Marketplace and get this error:

failed to install release: template: kong/templates/ingress-class.yaml:2:34: executing "kong/templates/ingress-class.yaml" at <lookup "networking.k8s.io/v1" "IngressClass" "" "kong">: error calling lookup: ingressclasses.networking.k8s.io "kong" is forbidden: User "system:serviceaccount:openshift-operators:kong-operator" cannot get resource "ingressclasses" in API group "networking.k8s.io" at the cluster scope

nautiam commented 2 years ago

I found a workaround to allow Kong-operator to create ingressclass. I create a ClusterRole "icadmin" with full permission to get and create ingressclasses and a ClusterRoleBinding "icadmin" to add "system:serviceaccount:openshift-operators:kong-operator" to "icadmin" ClusterRole. Then it works. I think that you may consider to add this step to create ClusterRole and ClusterRoleBinding to CRDs in next release of kong-operator.

$ oc apply -f rbac/kong-cluster-role.yaml
$ oc apply -f rbac/kong-clusterrolebinding.yaml

rbac.zip

shaneutt commented 2 years ago

Thank you for reporting this problem, and for posting your workaround.

Our expectation is that this should work with the latest release as the relevant permissions are present:

However we're investigating an issue that has occurred with upstream publishing to the Redhat Marketplace which may have had an effect on this. Let us know in the meantime if there are other questions or concerns we can address for you, otherwise we'll report back once we have confirmed resolution for the marketplace publishing issue.

rainest commented 2 years ago

@nautiam does this work using the version currently available at https://operatorhub.io/operator/kong ?

If so that should confirm that the latest permissions are correct and it's just an issue with the Marketplace copy of the package being out of date.

nautiam commented 2 years ago

@rainest I only tested with Kong from Marketplace of Openshift. I don't test with Kong from Operatorhub

rainest commented 2 years ago

Right, I mean can you test with the one from Operator Hub? While we're working out the issue with the Marketplace listing, it'd be useful to confirm if there's something actually wrong with the permission we added or if the Marketplace version was just missing the latest update.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

rainest commented 2 years ago

@nautiam we're still working with Red Hat regarding the status of the Marketplace listing. Had you had a chance to try with the Operator Hub version to see if it exhibits the issue still?

nautiam commented 2 years ago

@nautiam we're still working with Red Hat regarding the status of the Marketplace listing. Had you had a chance to try with the Operator Hub version to see if it exhibits the issue still?

Sorry I missed your last reply.

I tested kong-operator v0.9.0 from Operator Hub on Openshift and this issue still happens.

failed to install release: template: kong/templates/ingress-class.yaml:2:34: executing "kong/templates/ingress-class.yaml" at <lookup "networking.k8s.io/v1" "IngressClass" "" "kong">: error calling lookup: ingressclasses.networking.k8s.io "kong" is forbidden: User "system:serviceaccount:openshift-operators:kong-operator" cannot get resource "ingressclasses" in API group "networking.k8s.io" at the cluster scope

rainest commented 2 years ago

Ah, okay, I see now--we have it deploy/role.yaml, but when OLM handles the install, it generates the role from the CSV info, which is quite out of date: https://github.com/Kong/kong-operator/blob/main/olm/0.9.0/kong.v0.9.0.clusterserviceversion.yaml#L60-L130

cmwylie19 commented 2 years ago

There is a similar error when installing from the operator on OCP 4.10

{
  "level": "error",
  "ts": 1648670959.9781337,
  "logger": "controller.kong-controller",
  "msg": "Reconciler error",
  "name": "kong",
  "namespace": "kong",
  "error": "failed to install release: template: kong/templates/ingress-class.yaml:2:34: executing \"kong/templates/ingress-class.yaml\" at <lookup \"networking.k8s.io/v1\" \"IngressClass\" \"\" \"kong\">: error calling lookup: ingressclasses.networking.k8s.io \"kong\" is forbidden: User \"system:serviceaccount:openshift-operators:kong-operator\" cannot get resource \"ingressclasses\" in API group \"networking.k8s.io\" at the cluster scope",
  "stacktrace": "sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.10.0/pkg/internal/controller/controller.go:227"
}

I had worked around by assigning cluster-admin ClusterRole to the kong-operator ServiceAccount:

kubectl create clusterrolebinding kong-admin --clusterrole=cluster-admin --serviceaccount=openshift-operators:kong-operator

with the intent of narrowing the scope down more at a later time.

I like the proposed solution at the beginning of this thread as it is only scoped to the ingressclasses resource. https://github.com/Kong/kong-operator/issues/71#issuecomment-952527805

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

mflendrich commented 2 years ago

We're working on a release of Gateway Operator that a technical preview of is expected to come at the end of September.

From now on, Gateway Operator will supersede the Helm operator as per our deprecation notice.

pmalek commented 1 year ago

For those interested to learn about the Gateway Operator you can find that info:

As @mflendrich mentioned. This is marked as Technical Preview.