search

kube-logging / logging-operator

Logging operator for Kubernetes
https://kube-logging.dev
Apache License 2.0
1.52k stars 326 forks source link

Suggesting more restrictive k8s RBAC and PSP #662

Open bygui86 opened 3 years ago

bygui86 commented 3 years ago

Describe the bug: I think that RBAC and PSP applied to logging-operator, fluent-bit and fluentd are too permissive.

Proposal: I propose following RBAC and PSP to restrict as much as possible permissions to components, without jeopardizing or influencing normal functionalities:

operator RBAC + PSP:

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: logging-restricted
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: runtime/default
    seccomp.security.alpha.kubernetes.io/defaultProfileName:  runtime/default
    apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
    apparmor.security.beta.kubernetes.io/defaultProfileName:  runtime/default
spec:
  privileged: false
  allowPrivilegeEscalation: false
  requiredDropCapabilities:
    - ALL
  volumes:
    - configMap
    - secret
    - persistentVolumeClaim
    - emptyDir
    - projected
    - downwardAPI
  hostPID: false
  hostIPC: false
  hostNetwork: false
  runAsUser:
    rule: MustRunAsNonRoot
  runAsGroup:
    rule: MustRunAs
    ranges:
      - min: 100
        max: 65535
  supplementalGroups:
    rule: MustRunAs
    ranges:
      - min: 1
        max: 65535
  fsGroup:
    rule: MustRunAs
    ranges:
      - min: 1
        max: 65535
  seLinux:
    rule: RunAsAny
  readOnlyRootFilesystem: true
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: logging-operator
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: logging-operator
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: logging-operator
subjects:
  - kind: ServiceAccount
    name: logging-operator
    namespace: logging
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: logging-operator
rules:
  - apiGroups:
      - policy
    resources:
      - podsecuritypolicies
    verbs:
      - use
    resourceNames:
      - logging-restricted
  - apiGroups:
      - batch
    resources:
      - jobs
    verbs:
      - "*"
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: logging-operator
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: logging-operator
subjects:
  - kind: ServiceAccount
    name: logging-operator
    namespace: logging
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: logging-operator
rules:
  - apiGroups:
      - policy
    resources:
      - podsecuritypolicies
    verbs:
      - use
    resourceNames:
      - logging-restricted
  - apiGroups:
      - monitoring.coreos.com
    resources:
      - servicemonitors
      - podmonitors
      - prometheusrules
    verbs:
      - "*"
  - apiGroups:
      - apiextensions.k8s.io
    resources:
      - customresourcedefinitions
    verbs:
      - list
      - watch
  - apiGroups:
      - logging.banzaicloud.io
    resources:
      - "*"
    verbs:
      - "*"
  - apiGroups:
      - ""
    resources:
      - nodes
      - namespaces
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - pods
      - services
      - endpoints
      - configmaps
      - secrets
      - persistentvolumeclaims
    verbs:
      - "*"
  - apiGroups:
      - apps
    resources:
      - deployments
      - daemonsets
      - statefulsets
    verbs:
      - "*"
  - apiGroups:
      - ""
      - rbac.authorization.k8s.io
    resources:
      - serviceaccounts
      - roles
      - rolebindings
      - clusterroles
      - clusterrolebindings
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - policy
    resources:
      - podsecuritypolicies
    verbs:
      - get
      - list
      - watch

fluent-bit RBAC + PSP:

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: logging-fluentbit-restricted
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: runtime/default
    seccomp.security.alpha.kubernetes.io/defaultProfileName:  runtime/default
    apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
    apparmor.security.beta.kubernetes.io/defaultProfileName:  runtime/default
spec:
  privileged: false
  allowPrivilegeEscalation: false
  requiredDropCapabilities:
    - ALL
  hostPID: false
  hostIPC: false
  hostNetwork: false
  runAsGroup:
    rule: MustRunAs
    ranges:
      - min: 100
        max: 65535
  supplementalGroups:
    rule: MustRunAs
    ranges:
      - min: 1
        max: 65535
  fsGroup:
    rule: MustRunAs
    ranges:
      - min: 1
        max: 65535
  seLinux:
    rule: RunAsAny
  readOnlyRootFilesystem: true
  # DIFFERENCES from logging-restricted
  volumes:
    - configMap
    - secret
    - emptyDir
    - hostPath
  allowedHostPaths:
    # why this folder even considering it's empty? anyway read only...
    - pathPrefix: /var/lib/docker/containers
      readOnly: true
    - pathPrefix: /var/log
      readOnly: false
  runAsUser:
    rule: RunAsAny
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: logging-fluentbit
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: logging-fluentbit
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: logging-fluentbit
subjects:
- kind: ServiceAccount
  name: logging-fluentbit
  namespace: logging
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: logging-fluentbit
rules:
  - apiGroups:
      - policy
    resources:
      - podsecuritypolicies
    verbs:
      - use
    resourceNames:
      - logging-fluentbit-restricted
  - apiGroups:
    - ""
    resources:
    - pods
    - namespaces
    verbs:
    - get
    - list
    - watch

fluentd RBAC + PSP:

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: logging-fluentd-restricted
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: runtime/default
    seccomp.security.alpha.kubernetes.io/defaultProfileName:  runtime/default
    apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
    apparmor.security.beta.kubernetes.io/defaultProfileName:  runtime/default
spec:
  privileged: false
  allowPrivilegeEscalation: false
  requiredDropCapabilities:
    - ALL
  hostPID: false
  hostIPC: false
  hostNetwork: false
  runAsUser:
    rule: MustRunAsNonRoot
  runAsGroup:
    rule: MustRunAs
    ranges:
      - min: 100
        max: 65535
  supplementalGroups:
    rule: MustRunAs
    ranges:
      - min: 1
        max: 65535
  fsGroup:
    rule: MustRunAs
    ranges:
      - min: 1
        max: 65535
  seLinux:
    rule: RunAsAny
  # DIFFERENCES from logging-restricted
  volumes:
    - configMap
    - secret
    - persistentVolumeClaim
    - emptyDir
    - hostPath
  # readOnlyRootFilesystem: true
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: logging-fluentd
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: logging-fluentd
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: logging-fluentd
subjects:
  - kind: ServiceAccount
    name: logging-fluentd
    namespace: logging
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: logging-fluentd
rules:
  - apiGroups:
      - policy
    resources:
      - podsecuritypolicies
    verbs:
      - use
    resourceNames:
      - logging-fluentd-restricted
  - apiGroups:
      - ""
    resources:
      - configmaps
      - secrets
    verbs:
      - "*"

logging usage example:

apiVersion: logging.banzaicloud.io/v1beta1
kind: Logging
metadata:
  name: logging-components
spec:
  enableRecreateWorkloadOnImmutableFieldChange: true
  controlNamespace: logging
  fluentbit:
    metrics:
      serviceMonitor: true
    tolerations:
      - key: node-role.kubernetes.io/master
        operator: Exists
        effect: NoSchedule
      - operator: Exists
        effect: NoExecute
      - operator: Exists
        effect: NoSchedule
    security:
      roleBasedAccessControlCreate: false
      podSecurityPolicyCreate: false
      serviceAccount: logging-fluentbit
      podSecurityContext:
        fsGroup: 101
  fluentd:
    scaling:
      replicas: 3
    metrics:
      serviceMonitor: true
    livenessDefaultCheck: true
    security:
      roleBasedAccessControlCreate: false
      podSecurityPolicyCreate: false
      serviceAccount: logging-fluentd
      securityContext:
        readOnlyRootFilesystem: false
      podSecurityContext:
        runAsUser: 100
        runAsGroup: 100
        fsGroup: 101

Environment details: Above proposal were tested on an environment with following details:

Looking forward your feedback!!

/kind bug

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

stale[bot] commented 9 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions!

pepov commented 1 month ago

removed the help wanted label as: