search

postfinance / kubenurse

Kubernetes network monitoring
MIT License
416 stars 39 forks source link

clusterrole permission #17

Closed guleng closed 3 years ago

guleng commented 3 years ago

hi @djboris9 After I deploy version 3.0, I will prompt an error. Even if the clusterrole permission is given to admin, it will not work

E0415 06:59:52.669811       1 reflector.go:127] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:kube-system:nurse" cannot list resource "nodes" in API group "" at the cluster scope
E0415 06:59:53.957503       1 reflector.go:127] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:kube-system:nurse" cannot list resource "nodes" in API group "" at the cluster scope
E0415 06:59:57.066479       1 reflector.go:127] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:kube-system:nurse" cannot list resource "nodes" in API group "" at the cluster scope
E0415 07:00:02.396648       1 reflector.go:127] pkg/mod/k8s.io/client-go@v0.19.4/tools/cache/reflector.go:156: Failed to watch *v1.Node: failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:kube-system:nurse" cannot list resource "nodes" in API group "" at the cluster scope
djboris9 commented 3 years ago

Hi @guleng

Thanks for this bug report. We implemented a nodeCache that watches Nodes resources and therefore needs additional permissions to access them. This is done in order to filter out neighboring kubenurses on nodes that are unschedulable.

I will implement a flag to disable this (new) feature and provide an updated RBAC definition

djboris9 commented 3 years ago

Can you try it with the following additional RBAC resources?

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kubenurse-node-watcher
rules:
- apiGroups: [""]
  resources: ["nodes"]
  verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: nurse-node-watcher
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kubenurse-node-watcher
subjects:
- kind: ServiceAccount
  name: nurse
  namespace: kube-system

I'm just preparing/testing https://github.com/postfinance/kubenurse/commit/cd9ac29bfdec070a374ca44f41d3a03f466c8607 which will allow to disable node resource access.

guleng commented 3 years ago

I tried and the result was the same I got the same results on kubernetes 1.16 and 1.14 I'll wait for the next version

djboris9 commented 3 years ago

Ok, next week the new version should be out. I just saw a mistake in the snipped I provided. It should be like this sample below.

Together with the new release, there will be automated CI testing using the provided examples so we can spot such issues earlier.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kubenurse-node-watcher
rules:
- apiGroups: [""]
  resources: ["nodes"]
  verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: nurse-node-watcher
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kubenurse
subjects:
- kind: ServiceAccount
  name: nurse
  namespace: kube-system
djboris9 commented 3 years ago

Can you please try v1.3.4: https://github.com/postfinance/kubenurse/releases/tag/v1.3.4 Also using the new example RBACs