Request: cgroup v2 node pool in community clusters

kubernetes / test-infra

Test infrastructure for the Kubernetes project.

Apache License 2.0

3.82k stars 2.64k forks source link

Request: cgroup v2 node pool in community clusters #31572

Closed AkihiroSuda closed 2 months ago

AkihiroSuda commented 8 months ago

Currently k8s-infra-prow-build and eks-prow-build-cluster are still using cgroup v1, although many users should have already switched to cgroup v2. It is still possible to create a custom cluster with Boskos, but this is often complicated and hard to test on laptop.

Any chance to have cgroup v2 node pool in community clusters, or just have a new cluster dedicated for cgroup v2?

AkihiroSuda commented 8 months ago

/sig testing

ameukam commented 8 months ago

cc @BenTheElder

kannon92 commented 8 months ago

In a sig-node meeting a few weeks back, we did talk about making cgroups v2 the default. cc @dchen1107 @SergeyKanzhelev

It is possible to get a cgroupv2 cluster but you may have to target it specifically. We have examples of cgroupv2 jobs.

BenTheElder commented 8 months ago

This is also /sig k8s-infra

We have https://github.com/kubernetes/k8s.io/issues/5276

In a sig-node meeting a few weeks back, we did talk about making cgroups v2 the default. cc @dchen1107 @SergeyKanzhelev

It is possible to get a cgroupv2 cluster but you may have to target it specifically. We have examples of cgroupv2 jobs.

This issue is about the cluster on which we run the prowjob pods, so that nested local clusters (like KIND, hack/local-up-cluster.sh) run on cgroupsv2.

We could create a cgroupsv2 node-pool or cluster on GKE and run some jobs there ^1, someone else from k8s infra will have to chime in for the EKS clusters. @xmudrii

dims commented 5 months ago

fyi all the ci jobs wiht al2023 in their names are cgroupsv2

k8s-triage-robot commented 2 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

BenTheElder commented 2 months ago

/remove-lifecycle stale

We're still going to want to sort this out at some point, eventually this will be the default even ...

dims commented 2 months ago

we have a lot of cgroupv2 jobs now ... https://prow.k8s.io/?job=*cgroupv2*

BenTheElder commented 2 months ago

we have a lot of cgroupv2 jobs now ... https://prow.k8s.io/?job=*cgroupv2*

Yes, and that's probably more than enough for now, but at some point the prow clusters will also wind up migrating, and we could make sure kind etc are tested that way.

For now, I guess the cloudy E2E are enough and we can track this https://github.com/kubernetes/k8s.io/issues/5276 instead