search

canonical / bundle-kubeflow

Charmed Kubeflow
Apache License 2.0
102 stars 49 forks source link

rbac has to be turned off, or the istio-ingressgateway-operator role has to be manually modified #268

Closed jhobbs closed 2 years ago

jhobbs commented 3 years ago

This should work with rbac on without any extra role config.

Deadleg commented 3 years ago

Adding a bit more detail; the operator logs this error:

Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Audit-Id': '12770c51-a979-41d1-b453-1bda3f9013aa', 'Cache-Control': 'no-cache, private', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff', 'Date': 'Tue, 11 May 2021 02:27:20 GMT', 'Content-Length': '374'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"configmaps \"istio-ca-root-cert\" is forbidden: User \"system:serviceaccount:kubeflow:istio-ingressgateway-operator\" cannot get resource \"configmaps\" in API group \"\" in the namespace \"kubeflow\"","reason":"Forbidden","details":{"name":"istio-ca-root-cert","kind":"configmaps"},"code":403}

Which requires the istio-ingressgateway-operator role to be updated with get permissions for configmaps.

DomFleischmann commented 3 years ago

On which version are you testing this?

Deadleg commented 3 years ago

I ran this yesterday on a fresh EKS 1.19 cluster using the latest full kubeflow bundle (juju deploy cs:kubeflow). I was following the install instructions on the website so no customisations were made.

DomFleischmann commented 3 years ago

@Deadleg okay so, the issue that @jhobbs filed should be resolved by now, I think your issue might be specific to EKS as we don't actively test against EKS. If you have the option of deploying on some other kubernetes like AKS, Charmed Kubernetes or Microk8s you will probably get better results.

Eitherway a new issue for this bug on EKS should probably be opened.

DomFleischmann commented 3 years ago

Currently with RBAC the following manual step is required: kubectl patch role -n kubeflow istio-ingressgateway-operator -p '{"apiVersion":"rbac.authorization.k8s.io/v1","kind":"Role","metadata":{"name":"istio-ingressgateway-operator"},"rules":[{"apiGroups":["*"],"resources":["*"],"verbs":["*"]}]}'

This is documented in our official documentation