cannot get resource "nodes" in API group "" at the cluster scope

[ghostsquad]

{"level":"error","error":"nodes \"<REDACTED>\" is forbidden: User \"system:serviceaccount:monitoring:event-exporter\" cannot get resource \"nodes\" in API group \"\" at the cluster scope","time":"2021-01-04T14:18:54Z","caller":"/app/pkg/kube/watcher.go:80","message":"Cannot list annotations of the object"}

k get clusterrole view -o yaml
aggregationRule:
  clusterRoleSelectors:
  - matchLabels:
      rbac.authorization.k8s.io/aggregate-to-view: "true"
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
  creationTimestamp: "2019-04-17T18:14:02Z"
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
    rbac.authorization.k8s.io/aggregate-to-edit: "true"
  name: view
  resourceVersion: "425236435"
  selfLink: /apis/rbac.authorization.k8s.io/v1/clusterroles/view
  uid: 942388e0-613c-11e9-86c3-02ac0a6842ac
rules:
- apiGroups:
  - argoproj.io
  resources:
  - rollouts
  - rollouts/scale
  - experiments
  - analysistemplates
  - clusteranalysistemplates
  - analysisruns
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - configmaps
  - endpoints
  - persistentvolumeclaims
  - pods
  - replicationcontrollers
  - replicationcontrollers/scale
  - serviceaccounts
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - bindings
  - events
  - limitranges
  - namespaces/status
  - pods/log
  - pods/status
  - replicationcontrollers/status
  - resourcequotas
  - resourcequotas/status
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - apps
  resources:
  - controllerrevisions
  - daemonsets
  - deployments
  - deployments/scale
  - replicasets
  - replicasets/scale
  - statefulsets
  - statefulsets/scale
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - autoscaling
  resources:
  - horizontalpodautoscalers
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - batch
  resources:
  - cronjobs
  - jobs
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - extensions
  resources:
  - daemonsets
  - deployments
  - deployments/scale
  - ingresses
  - networkpolicies
  - replicasets
  - replicasets/scale
  - replicationcontrollers/scale
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - policy
  resources:
  - poddisruptionbudgets
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - networkpolicies
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - get
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - list
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - watch
- apiGroups:
  - metrics.k8s.io
  resources:
  - pods
  - nodes
  verbs:
  - get
  - list
  - watch

We are on Kubernetes version 1.15

Do you know if it's safe to edit the role, given that it's a default/built-in role in kubernetes? Or is there another problem?

[ghostsquad]

The other solution here is to provide an additional cluster role and rolebinding that grants access to nodes explicitly.

[skropf]

You can either add "nodes" to the mentioned API group or create your own ClusterRole.

I would go with creating your own ClusterRole so that you have full control over the event-exporter and you don't influence/mess around with other services using the view Role.

An example ClusterRole with which it should work:

kind: ClusterRole
metadata:
  name: kubernetes-event-exporter
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - nodes
  - pods
  - events
  - services
  - resourcequotas
  - replicationcontrollers
  - limitranges
  - persistentvolumeclaims
  - persistentvolumes
  - namespaces
  - endpoints
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - apps
  resources:
  - statefulsets
  - daemonsets
  - deployments
  - replicasets
  verbs:
  - get
  - list
  - watch

[ghostsquad]

Since you can bind multiple roles to a service account (and the permissions are additive) that's what I did.

This issue serves as a indicator that the manifests provided to install this should also add a role with the specific node permissions.

Then bind to the created role as well as the built-on role.