Migrate away from PodSecurityPolicy and PodDisruptionBudget which got deprecated in k8s 1.21

pmalek-sumo commented 3 years ago

Because of the deprecation of PodSecurityPolicy and PodDisruptionBudget in k8s 1.21 (k8s docs) users see the below notice when installing our chart:

W0910 12:35:50.206929    5318 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0910 12:35:50.208431    5318 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0910 12:35:50.209579    5318 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0910 12:35:50.210599    5318 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0910 12:35:50.212232    5318 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0910 12:35:50.220400    5318 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0910 12:35:50.222055    5318 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0910 12:35:50.223532    5318 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
W0910 12:35:50.224911    5318 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
yW0910 12:36:51.040452    5318 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0910 12:36:51.040659    5318 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0910 12:36:51.041005    5318 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0910 12:36:51.041338    5318 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0910 12:36:51.041707    5318 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0910 12:36:51.044525    5318 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0910 12:36:51.044673    5318 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
W0910 12:36:51.052004    5318 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
W0910 12:36:51.052004    5318 warnings.go:70] policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget

We should mitigate that by following the guidelines from k8s authors and maintainers. Most likely we'll need to look into "PSP Replacement Policy".

Here's a note from the above linked blog post mentioning this:

Kubernetes SIG Security, SIG Auth, and a diverse collection of other community members have been working together for months to ensure that what’s coming next is going to be awesome. We have developed a Kubernetes Enhancement Proposal (KEP 2579) and a prototype for a new feature, currently being called by the temporary name "PSP Replacement Policy." We are targeting an Alpha release in Kubernetes 1.22.

andrzej-stencel commented 2 years ago

As a workaround, to install the chart in Kubernetes v1.25.x which removed the PodSecurityPolicy completely, apply the following change to the values.yaml file:

kube-prometheus-stack:
  global:
    rbac:
      pspEnabled: false
  kube-state-metrics:
    podSecurityPolicy:
      enabled: false
  prometheus-node-exporter:
    rbac:
      pspEnabled: false

rnishtala-sumo commented 1 year ago

Considering this deprecation:

Should we remove podSecurityPolicy for fluend resources? (since we're also switching to otc by default)
The pdb api version has already been set to policy/v1 here
- https://github.com/SumoLogic/sumologic-kubernetes-collection/blob/main/deploy/helm/sumologic/templates/_api_versions_helpers.tpl#L10
kube-prometheus-stack disables PodSecurityPolicies by default because they are deprecated in Kubernetes 1.21 and will be removed in Kubernetes 1.25, starting version 27.x. We're at 39.x

Considering this, it looks like we only need to remove podSecurityPoliciy for fluend resources.

rnishtala-sumo commented 1 year ago

This issue has been addressed here: https://github.com/SumoLogic/sumologic-kubernetes-collection/commit/5353114ed3f13165e470bf0860f5891df788e04e

andrzej-stencel commented 1 year ago

This issue shouldn't have been closed. It can be closed when the chart v2 can be installed into Kubernetes v1.25 without any customizations. Currently installation fails, see https://github.com/SumoLogic/sumologic-kubernetes-collection/issues/2729.

andrzej-stencel commented 1 year ago

This was discussed in the team again and the outcome is:

This issue is fixed in v3 of the collection chart.
This issue cannot be fixed in v2 without a breaking change. The current decision is to not introduce this breaking change and to not declare support for Kubernetes v1.25 and higher with the chart v2.

phelian commented 1 year ago

@astencel-sumo Thank you for updating us, however, I wonder about the statement of 1.25 and above since the current support matrix (https://github.com/SumoLogic/sumologic-kubernetes-collection/blob/release-v2.19/deploy/README.md#support-matrix) stops at 1.23 (EKS) When I asked in the sumo slack I was referred to your workaround for EKS 1.24 support, if there other items that are blocking 1.24?

andrzej-stencel commented 1 year ago

I have posted in the Slack thread. EKS v1.24 is coming soon :crossed_fingers: :slightly_smiling_face:

SumoLogic / sumologic-kubernetes-collection

Migrate away from PodSecurityPolicy and PodDisruptionBudget which got deprecated in k8s 1.21 #1742