search

stackabletech / issues

This repository is only for issues that concern multiple repositories or don't fit into any specific repository
2 stars 0 forks source link

Evaluate Kubernetes 1.30 changes #618

Closed nightkr closed 3 months ago

nightkr commented 3 months ago

(Template shamelessly copied from #460)

I would like to know if there are any changes in Kubernetes 1.30 that the SDP could potentially make use of (e.g. sidecars), will require changes or will break things on our end.

This should be timeboxed and take up at most 4h of research and reading. The result of this should be a comment on this issue or follow-up issues listing the high level points of things we could, should and should not do.

Must read:

nightkr commented 3 months ago

Breaking Changes

Volume mode may not be changed during restore

Shouldn't affect us per se since I don't believe we use volume snapshots, but may catch some customers? Worth keeping an eye open for.

Deprecations

None of note.

Interesting new features

Node Log Query

Beta, disabled by default.

Not super useful for SDP, but may (over time) make life easier when debugging.

Could also consider if we want to encourage people to set up Vector rules to collect it, at some point?

CRD Validation Ratcheting

Beta, enabled by default.

Some validation is no longer enforced on unmodified fields in the CRD. This would help mitigate some of the breakage from tightening up our CRD validation over time.

On the other hand, breaking changes would still break new installations and restores.

Job Success Policy

Alpha, disabled by default.

Allows indicating that only certain job pod indexes are relevant for job success, I think? Unclear how useful this is.

Traffic Distribution Preferences

Alpha, disabled by default.

Allows Services to request that traffic is directed as close as possible but with best-effort, contrary to internalTrafficPolicy: Local which will drop traffic that has no local destination.

This could be useful for cases like OPA in order to maintain availability while still minimizing network overhead. However, persistent connections may end up pinning traffic to a remote node even once local service is restored.

Storage Version Migration

Alpha, disabled by default.

Allows us to request explicit migration eagerly, rather than waiting for the object to be modified. Should help us remove deprecated CRD versions eventually.

CEL Admission Control

Stable, always enabled.

This allows users to manage validation rules in Kubernetes itself, without having to shell out to an external policy layer like OPA (which is often configured to fail-open, allowing race conditions where writes can bypass the policy if OPA is down).

Immediately I can see this being useful for enforcing rules like "only allow SecretClass foo to be used from Namespaces bar and baz".

NickLarsenNZ commented 3 months ago

Job Success Policy Allows indicating that only certain job pod indexes are relevant for job success, I think? Unclear how useful this is.

This could be handy when you have sidecars that are best efforts (There was a time I could have found this useful)

sbernauer commented 3 months ago

The list looks good to me, IMHO we can close the issue. For me the only thing we could do something right now would be CEL Admission Control

nightkr commented 3 months ago

Job Success Policy Allows indicating that only certain job pod indexes are relevant for job success, I think? Unclear how useful this is.

This could be handy when you have sidecars that are best efforts (There was a time I could have found this useful)

Sidecars are pod-internal, this is for ignoring some pods, I think?

sbernauer commented 3 months ago

Sidecars are pod-internal, this is for ignoring some pods, I think?

Yep, AFAIK this has nothing to do with sidecars