Closed imjasonh closed 3 years ago
imjasonh
commented
3 years ago kind cluster create accepts a config file which can take kubeadm config patches, and kubeadm config can set apiserver flags to write audit logs to a file (newline-delimited JSON objects)
imjasonh
commented
3 years ago /tmp/audit/policy.yaml:
apiVersion: audit.k8s.io/v1beta1
kind: Policy
rules:
- level: Metadata
stages:
- ResponseCompleteconfig.yaml:
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
extraMounts:
- hostPath: /tmp/audit
containerPath: /etc/kubernetes/pki/audit
readOnly: False
kubeadmConfigPatches:
- |
kind: ClusterConfiguration
metadata:
name: config
apiServer:
extraArgs:
"audit-policy-file": "/etc/kubernetes/pki/audit/policy.yaml"
"audit-log-path": "/etc/kubernetes/pki/audit/audit.log"Create the KinD cluster:
kind create cluster --name=audit --config=config.yaml[not pictured]
The KinD cluster's control plane node writes newline-delimited JSON objects to /tmp/audit/audit.log, which you can process with jq, grep, etc.
cat /tmp/audit/audit.log | jq '. | "\(.objectRef.apiGroup)/\(.objectRef.apiVersion) \(.objectRef.resource) \(.verb) \(.objectRef.namespace) \(.user.username)"' | grep tekton-pipelines-controller\" | sort -u It probably makes sense to separate these requests into buckets for "my namespace(s)" (tekton-pipelines) and "other namespaces" (arendelle-zrstn), to tell whether it should be granted by a namespaced Role or a ClusterRole.
imjasonh
commented
3 years ago Of course after I wrote a bunch of code I find out this already exists 🤦♂️
https://github.com/alcideio/rbac-tool#rbac-tool-auditgen
based on
https://github.com/liggitt/audit2rbac https://www.youtube.com/watch?v=n2cD20moYe8
The KinD+e2e tests part seems novel at least, I can hook that up to these tools.
imjasonh
commented
3 years ago
When writing a complex k8s controller it can be difficult to fully catalog what permissions it needs to run. You might just request a bunch of access you don't need and be content that it runs successfully. But that's bad.
Instead, it'd be cool to have a tool that ran some e2e tests while recording what the controller does.
Roles and/orClusterRoles describing all the stuff the controller did during the testskubectl-rbacwhich is archived now, and this Medium article describing it.