search

groundcover-com / caretta

Instant K8s service dependency map, right to your Grafana.
Apache License 2.0
1.79k stars 67 forks source link

Support for Kubernetes versions 1.21 and above #49

Closed cmcmullan94 closed 1 year ago

cmcmullan94 commented 1 year ago

Is your feature request related to a problem? Please describe. I would like to sample caretta across some nodes I have running on kube v1.24.6, however, I currently cannot deploy as Caretta wants to use PodSecurityPolicy. This has been deprecated in Kube versions 1.21 and has been removed 1.25. Kubernetes recommend migrating from the PodSecurityPolicy to the built in PodSecurity Admission Controller - here

Describe the solution you'd like Groundcover to apply fix by enabling support for kube versions 1.21 and greater

Describe alternatives you've considered No alternatives

Additional context Error Output

ryechezkel commented 1 year ago

Hi @cmcmullan94 , can you explain a bit more about the problem ? You said you are running v1.24 using psp should still work.

Can you also try adding --set rbac.pspEnabled=false and see if it works?

This will deploy without rbac

cmcmullan94 commented 1 year ago

Hi @ryechezkel - yes, I am running Kube v1.24.6. With rbac.pspEnabled = false, the same error as above still persists. The kube documentation linked in the original post, notes that PSP has been deprecated from v1.21 and removed from v1.25. I have just found this doc, from April 2021 which states that PSP "will pass fully out of support for all kubernetes versions" after nearly 2 years. This document is from April 2021, could I be right in saying this is the case as we are currently 2 months short of that 2 year mark ?

As mentioned, my error message is above and has not changed with pspEnabled = false.

noamApps commented 1 year ago

hey @cmcmullan94 , I'll check why the problem persist even though rbac.pspEnabled=false.

regarding the PSP matter in general, eventually, we are definitely going to remove PSP from the chart, unfortunately there is more to this deprecation than meets the eye, the Pod admission controller is not a drop in replacement, it does not support pod-level security boundaries out of the box (only namespace level) which is not suitable for scenarios with agent/backend requiring different boundaries.

Since security is a top concern for us, we carefully design the next iteration of security boundaries, meanwhile, we decided to deploy PSP by default in v<=1.24 as it will work out of the box for all users, instead of disabling it by default, which will cause the installation to fail for users with v<1.24 && rbac enabled

I'll investigate the pspEnabled=false problem promptly and get back to you.

noamApps commented 1 year ago

Hey @cmcmullan94, sorry for the delayed response The VictoriaMetrics sub-chart also automatically deploys PSP objects (probably for the same reason as us) So in order to disable PSP deployment completely, use --set rbac.pspEnabled=false,victoria-metrics-single.rbac.pspEnabled=false

I ran helm template to make sure no PSPs are being deployed, but let me know if you run into further issues.

cmcmullan94 commented 1 year ago

Hi @noamApps - thanks for the update, this has resolved my issue!

Thanks again!