Prombench `cluster-infra` crashes on applying manifest files with Kind Provider

[Vandit1604]

https://github.com/prometheus/test-infra/blob/master/prombench/manifests/cluster-infra/2_ingress-nginx-controller.yaml is continuously crashing when I try to apply the manifest files with the ../infra/infra with Kind provider. Is it because of it being ancient (last updated ~4 years ago)
It's documentation also needs to be updated. I'm willing to help if someone can help me when I get blocked due to some issue I face and can't able to figure out due to lack of context.

[Vandit1604]

Must've accidentally closed it. :^)

[Vandit1604]

I went through the logs of failing container.

kubectl logs pod/ingress-nginx-admission-patch-wfck6 -n ingress-nginx

W0801 13:51:36.072559       1 client_config.go:618] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
{"level":"info","msg":"patching webhook configurations 'ingress-nginx-admission' mutating=false, validating=true, failurePolicy=Fail","source":"k8s/k8s.go:118","time":"2024-08-01T13:51:36Z"}
{"err":"validatingwebhookconfigurations.admissionregistration.k8s.io \"ingress-nginx-admission\" not found","level":"fatal","msg":"failed getting validating webhook","source":"cmd/patch.go:103","time":"2024-08-01T13:51:36Z"}

[Vandit1604]

https://github.com/prometheus/test-infra/blob/b2a0134637628a720fc5303bcb2de31f7275545b/prombench/manifests/cluster-infra/4_kube-state-metrics.yaml#L101-L104

kubectl logs pod/kube-state-metrics-8544665d78-ttmr5 -n kube-system
Error: unknown flag: --collectors
Usage:
  kube-state-metrics [flags]
  kube-state-metrics [command]

Available Commands:
  completion  Generate completion script for kube-state-metrics.
  help        Help about any command
  version     Print version information.

What meterics do we want from kube-state-meterics here? We can update the kube-state-metrics image to v2.13.0 and write the configuration flag according to that.

[yomek33]

I believe there may be challenges in various areas 😂 The issue I'm currently encountering is an error stating that IngressClass is not implemented.

00:37:39 k8s.go:314: resource updated - kind: ClusterRoleBinding, name: ingress-nginx-admission
00:37:39 k8s.go:358: resource updated - kind: ConfigMap, name: ingress-nginx-controller
00:37:39 k8s.go:836: resource updated - kind: Service, name: ingress-nginx-controller
00:37:39 k8s.go:836: resource updated - kind: Service, name: ingress-nginx-controller-admission
00:37:39 k8s.go:442: resource updated - kind: Deployment, name: ingress-nginx-controller
00:37:49 provider.go:74: Request for 'applying deployment:ingress-nginx-controller' is done!
00:37:49 k8s.go:538: resource created - kind: Job, name: ingress-nginx-admission-create
00:37:59 provider.go:74: Request for 'running job:ingress-nginx-admission-create' is done!
00:37:59 k8s.go:538: resource created - kind: Job, name: ingress-nginx-admission-patch
00:38:09 provider.go:74: Request for 'running job:ingress-nginx-admission-patch' is done!
Error parsing commandline arguments: error applying 'manifests/cluster-infra/2_ingress-nginx-controller.yaml' err:creating request for unimplimented resource type:ingressclass

However, since it's mentioned here, I believe the problem lies not in the YAML but in the Go code. https://github.com/prometheus/test-infra/blob/master/prombench/manifests/cluster-infra/2_ingress-nginx-controller.yaml#L621-L634

but if this is an issue specific to my environment (M2 Mac), it could be due to problems with my other tools.

[Vandit1604]

I think It is code related problem in infra/infra for the Kind provider.

[Vandit1604]

I'm investigating that.

[yomek33]

I guess we should add case "inglessClass" to here🤔https://github.com/prometheus/test-infra/blob/34a189da3778d9035d430e3213f5c9dd18d520a7/pkg/provider/k8s/k8s.go#L146-L154

[Vandit1604]

It was failing for me as well. I have implemented the fix locally, but then it is failing for

https://github.com/prometheus/test-infra/blob/b2a0134637628a720fc5303bcb2de31f7275545b/prombench/manifests/cluster-infra/4_kube-state-metrics.yaml#L101-L104

[Vandit1604]

@yomek33 I can raise a PR for the fixes I found till now. I was thinking of doing that in single PR but since a lot of people are trying out the project, dividing it in smaller PRs feels right.

[yomek33]

Yes, that sounds great! It will be really helpful if you create the PRs😊

[Vandit1604]

Hi @yomek33, I have created https://github.com/prometheus/test-infra/issues/717

[Vandit1604]

It was failing for me as well. I have implemented the fix locally, but then it is failing for

https://github.com/prometheus/test-infra/blob/b2a0134637628a720fc5303bcb2de31f7275545b/prombench/manifests/cluster-infra/4_kube-state-metrics.yaml#L101-L104

here

 - name: kube-state-metrics 
   image: registry.k8s.io/kube-state-metrics/kube-state-metrics:v2.10.1 
   args: 
   - "--resources=namespaces" 

made it work for me.