elastic / cloud-on-k8s

Elastic Cloud on Kubernetes
Other
2.56k stars 695 forks source link

Restricted Installation instructions incomplete/broken #7747

Open kfox1111 opened 4 months ago

kfox1111 commented 4 months ago

Bug Report

What did you do?

Followed the instructions here: https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-install-helm.html#k8s-install-helm-restricted

What did you expect to see?

They work

What did you see instead? Under which circumstances?

The operator chart couldn't be installed as a user with a rolebinding of admin in their namespace.

Two problems exist.

  1. There isnt the ClusterRoles for elastic-operator-edit and elastic-operator-view by the process. I manually rendered them from the chart and loaded them in.

  2. Even then, the admin user in the namespace didn't have enough permissions:

    APIGroups:["agent.k8s.elastic.co"], Resources:["agents/finalizers"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["agent.k8s.elastic.co"], Resources:["agents/status"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["apm.k8s.elastic.co"], Resources:["apmservers/finalizers"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["apm.k8s.elastic.co"], Resources:["apmservers/status"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["authorization.k8s.io"], Resources:["subjectaccessreviews"], Verbs:["create"]}
    {APIGroups:["autoscaling.k8s.elastic.co"], Resources:["elasticsearchautoscalers/finalizers"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["autoscaling.k8s.elastic.co"], Resources:["elasticsearchautoscalers/status"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["beat.k8s.elastic.co"], Resources:["beats/finalizers"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["beat.k8s.elastic.co"], Resources:["beats/status"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["elasticsearch.k8s.elastic.co"], Resources:["elasticsearches/finalizers"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["elasticsearch.k8s.elastic.co"], Resources:["elasticsearches/status"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["enterprisesearch.k8s.elastic.co"], Resources:["enterprisesearches/finalizers"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["enterprisesearch.k8s.elastic.co"], Resources:["enterprisesearches/status"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["kibana.k8s.elastic.co"], Resources:["kibanas/finalizers"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["kibana.k8s.elastic.co"], Resources:["kibanas/status"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["logstash.k8s.elastic.co"], Resources:["logstashes/finalizers"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["logstash.k8s.elastic.co"], Resources:["logstashes/status"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["maps.k8s.elastic.co"], Resources:["elasticmapsservers/finalizers"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["maps.k8s.elastic.co"], Resources:["elasticmapsservers/status"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["stackconfigpolicy.k8s.elastic.co"], Resources:["stackconfigpolicies/finalizers"], Verbs:["get" "list" "watch" "create" "update" "patch"]}
    {APIGroups:["stackconfigpolicy.k8s.elastic.co"], Resources:["stackconfigpolicies/status"], Verbs:["get" "list" "watch" "create" "update" "patch"]}

    The SubjectAccessReview one is more sensitive then the rest. Is it really needed?

Environment

thbkrkr commented 3 months ago

The operator chart couldn't be installed as a user with a rolebinding of admin in their namespace.

I'm not sure I understand. Could you share the exact steps to reproduce the issue?

There isnt the ClusterRoles for elastic-operator-edit and elastic-operator-view by the process. I manually rendered them from the chart and loaded them in.

There is no ClusterRoleBinding because createClusterScopedResources is set to false. Instead there are elastic-operator roles created in each namespace (and associated rolebindings to bind the role to the elastic-system/elastic-operator service account).

> k get role -A | grep elastic
elastic-system   elastic-operator                                  2024-05-14T12:45:21Z
namespace-a      elastic-operator                                  2024-05-14T12:45:20Z
namespace-b      elastic-operator                                  2024-05-14T12:45:20Z
kfox1111 commented 3 months ago

Here's a bit of psudocode as each cluster is a little different. but should give you the general idea. if not, please let me know and I can try and make an even more concrete example.

kubeadm kubeconfig user --client-name=foo > foo.kubeconfig
kubectl create namespace foo
kubectl create rolebinding admin -n foo --clusterrole=admin --user foo

export KUBECONFIG=foo.kubeconfig

helm upgrade --install --version 2.12.1 -n foo eck-operator eck-operator \
  --set managedNamespaces=foo \
  --set installCRDs=false \
  --set createClusterScopedResources=false \
  --set webhook.enabled=false \
  --set config.validateStorageClass=false \
   --repo https://helm.elastic.co