nri-plugin can be deployed but requires installation of dbus in machine--local--local pod in the followin way, before enabling plugin:
# create shoot without extension
kubectl apply -f ./examples/shoot.yaml # ../gardener-extension-cri-resmgr/examples/shoot.yaml - extension needs to be disabled
# (or disable extension manually)
kubectl patch shoot local -n garden-local -p '{"spec":{"extensions": [ {"type": "cri-resmgr-extension", "disabled": true} ] } }'
# then install and check dbus service in machine shoot--local--local pod
kubectl exec -i -n shoot--local--local `kgp -n shoot--local--local -l app=machine -o name` -- bash -c "apt -y update ; apt -y install dbus; rm -rf /run/dbus/system_bus_socket ; systemctl start dbus.socket dbus.service; systemctl statu
# and then enable extension for shoot
kubectl patch shoot local -n garden-local -p '{"spec":{"extensions": [ {"type": "cri-resmgr-extension", "disabled": false} ] } }'
# then the end result is following
cd gardener
./hack/usage/generate-admin-kubeconf.sh > admin-kubeconf.yaml
KUBECONFIG='admin-kubeconf.yaml' kubectl get pods -n kube-system -l app.kubernetes.io/instance=balloons -o wide
KUBECONFIG='admin-kubeconf.yaml' kubectl logs -n kube-system -l app.kubernetes.io/instance=balloons
Defaulted container "nri-resource-policy-balloons" out of: nri-resource-policy-balloons, patch-runtime (init)
I: [ policy ] assigning container kube-system/coredns-5c4c97c6f-zpctw/coredns to balloon reserved[0]{cpus:"0", mems:"0"}
I: [ policy ] assigning container kube-system/nri-resource-policy-balloons-8nkg5/nri-resource-policy-balloons to balloon reserved[0]{cpus:"0,48", mems:"0"}
I: [ policy ] assigning container kube-system/calico-node-vertical-autoscaler-6cf9c8bf8d-gmnjk/autoscaler to balloon reserved[0]{cpus:"0,48", mems:"0"}
I: [ policy ] assigning container kube-system/calico-kube-controllers-56f86cc77b-7dlp5/calico-kube-controllers to balloon reserved[0]{cpus:"0,48", mems:"0"}
I: [ policy ] assigning container kube-system/blackbox-exporter-6d5b5499b-f8wfq/blackbox-exporter to balloon reserved[0]{cpus:"0,48", mems:"0"}
I: [ policy ] assigning container kube-system/calico-typha-horizontal-autoscaler-78ff9f78d8-kxrf9/autoscaler to balloon reserved[0]{cpus:"0,48", mems:"0"}
I: [ policy ] assigning container kube-system/apiserver-proxy-g49kl/sidecar to balloon reserved[0]{cpus:"0,48", mems:"0"}
I: [ policy ] assigning container kube-system/apiserver-proxy-g49kl/proxy to balloon reserved[0]{cpus:"0,48", mems:"0"}
I: [ policy ] assigning container kube-system/calico-node-89vwz/calico-node to balloon reserved[0]{cpus:"0,48", mems:"0"}
I: [ nri-plugin ] <= Synchronize
Bug,Issues,Things to do:
[ ] Gardener kind-based setup for E2E locally works only with cgroup v2 (e.g. SLES15SP6) with systemd driver (for docker two containerd's) - to be reported to Gardener team or containerd in machine pod needs to changed to cgroupfs!
[ ] dbus is required by NRI plugin init container - find a way to automate deployment of dbus service in machine pod some pods inside shoot worker node already try to map /run/dbus/system_bus_socket and it becomes directory which need to be deleted before starting dbus.service/socket with rm -rf /run/dbus/system_bus_socket
[ ] Consider renaming plugin (cri-resmgr to nri)
[ ] Support image vector (providing images in Gardener native way)
[ ] Config propagation (current config aren't considered by NRI, require configuration in a form of custom resources "Balloons Policy"
[ ] Consider supporting other nri-plugins: like memtierd, topology aware ...
Just for testing purposes.
Status as of 7/16/2024:
nri-plugin can be deployed but requires installation of dbus in machine--local--local pod in the followin way, before enabling plugin:
Bug,Issues,Things to do:
rm -rf /run/dbus/system_bus_socket