Open YTGhost opened 1 year ago
starnop
commented
1 year ago So far we've shown that it works by using Kyverno to support image policy, and there are two more issues that need to be addressed
YTGhost
commented
1 year ago @starComingup Regarding the supportability of kyverno for k0s/k3s, I verified that it works well. cc @Starnop
justadogistaken
commented
1 year ago @YTGhost Can I look Kyverno in this way? That is Kyverno can help us make it easy to manage k8s webhook?
YTGhost
commented
1 year ago @YTGhost Can I look Kyverno in this way? That is Kyverno can help us make it easy to manage k8s webhook?
@justadogistaken Yes, kyverno makes it easy to create webhooks dynamically by creating policies. The following is an example of a Policy that replaces image when creating or updating a Pod:
apiVersion : kyverno.io/v1
kind: ClusterPolicy
metadata:
name: redirect-registry
spec:
background: false
rules:
- name: prepend-registry-containers
match:
resources:
kinds:
- Pod
preconditions:
all:
- key: "{{request.operation}}"
operator: In
value:
- CREATE
- UPDATE
mutate:
foreach:
- list: "request.object.spec.containers"
patchStrategicMerge:
spec:
containers:
- name: "{{ element.name }}"
image: "sea.hub:5000/{{ images.containers.{{element.name}}.path}}:{{images.containers.{{element.name}}.tag}}"
Issue Description
Background
The Proposal(Using image proxy shim to support container runtime neutrally) was discussed at an interim community meeting on November 9, 2022. This proposal proposes to use cri shim to implement image proxy, so that we don't need to change the docker logic.
During the meeting we discussed what might be a more elegant approach than implementing a cri shim, thus trying to achieve the results we want in a shorter path.
The current community discussion has resulted in two scenarios：
Sealer run/apply renders resources that meet certain specifications when submitting k8s resources
Support via k8s webhook
This Proposal will investigate the case of support through k8s webhook
Introduction
What is a Kubernetes Pod Policy
In fact, what we want to implement is a Kubernetes Pod Policy. It mean a set of mechanisms for ensuring validating and remediation controls over Pods and their attributes. Using this mechanism, we can redirect image pull requests when the Pod is created.
After researching the Kubernetes Pod Policy, there are three research targets as follows:
Pod Security Admission
With the upcoming deprecation and subsequent removal of Pod Security Policies (PSPs) in Kubernetes. Pod Security Admission is the new, albeit incomplete, successor. But it only checks Pods and does not support modifications to Pods, so it does not meet our needs.
Gatekeeper Introduction
Gatekeeper is an open-source project and collaboration between a number of companies including Google and Microsoft, later donated to the CNCF. Now in its third iteration, Gatekeeper is the Kubernetes-specific implementation of Open Policy Agent (OPA), a general-purpose policy engine. Because of the relationship between Open Policy Agent with Gatekeeper, the project is often written "OPA/Gatekeeper" to acknowledge these ties. Gatekeeper enables request validation and, most recently, mutation. A major defining characteristic of Open Policy Agent is its reliance upon the use of a specialized programming language called Rego in order to implement the logic necessary for policy decisions. Through Rego, OPA is designed to be broadly applicable to disparate software and not Kubernetes exclusively, as well as achieve a high level of logic execution.
Kyverno Introduction
Kyverno is an open-source project originally from Nirmata and later donated to the CNCF. Like Gatekeeper, Kyverno is a Kubernetes policy engine with validation and mutation abilities, however it adds resource generation in addition to, most recently, API object lookups. Unlike Gatekeeper, Kyverno was written from the start solely for use in Kubernetes. A major defining characteristic of Kyverno is its generation ability and the architecture which does not require specialized knowledge of a programming language to render policy decisions. Primarily as a reaction to this requirement of a specialized language, Kyverno differentiates itself from Gatekeeper through its model of simplicity.
Kyverno vs Gatekeeper
kubectl explain)Analysis
Gatekeeper Advantages
Gatekeeper Disadvantages
Kyverno Advantages
Kyverno Disadvantages
To summarize, if you need to use application policies across systems or if you need to implement highly complex policies, you can use Gatekeeper. Otherwise, Gatekeeper offers no benefit.
In our case, we only use it under kubernetes and need a simple strategy, so Kyverno is the best choice for us.
How Kyverno works
Admission Controllers
If you want to add a new Admission controller, you need to compile it with kube-apiserver and restart kube-apiserver, which is a tedious task.
Dynamic Admission Control
Two special admission controllers:
You can use the two admission controllers to configure some custom HTTP servers to provide webhooks and execute user-specific logic in these HTTP servers, thus extending the capabilities of the admission controller without recompiling kube-apiserver
Kyverno's Architecture
WebhookhandlesAdmissionReviewrequests from the Kubernetes API serverMonitorcomponent creates and manages required configurationsPolicyControllerwatches policy resources and initiates background scans based on the configured scan intervalGenerateControllermanages the lifecycle of generated resourcesType: feature request
Describe what feature you want
Additional context
Add any other context or screenshots about the feature request here.