:bug: Adjust e2e test after the repo structure has changed

[michal-gubricky]

What this PR does / why we need it:

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged): Fixes #133

Special notes for your reviewer:

Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.

TODOs:

• [ ] squash commits
• [ ] include documentation
• [ ] add unit tests

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/6388db0135d641b08335a616153d08df

:x: e2e-openstack-quick FAILURE in 9m 12s

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/a7539fc86cd04bdaa53cd5bff6e53f8a

:x: e2e-openstack-quick FAILURE in 28m 11s

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/407ff53b177949e2adbba0341c125927

:x: e2e-openstack-quick FAILURE in 15m 53s

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/839dfa3d30cf463681ea2d107c2e119c

:x: e2e-openstack-quick FAILURE in 20m 17s

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/39ba7849aa46481eb0fdad830a8006dc

:x: e2e-openstack-quick FAILURE in 18m 38s

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/824270ebefad4374bc8fa91397c62c24

:x: e2e-openstack-quick FAILURE in 24m 46s

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/338f8b5ee147432ca6a13c266f341cc0

:x: e2e-openstack-quick FAILURE in 22m 15s

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/0c70cb8f45ba49e9b5a0f05f4726ed0c

:x: e2e-openstack-quick FAILURE in 18m 42s

[michal-gubricky]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/0c70cb8f45ba49e9b5a0f05f4726ed0c

❌ e2e-openstack-quick FAILURE in 18m 42s

Obtained weird error:

error reconciling the Cluster topology: failed to create OpenStackMachineTemplate.infrastructure.cluster.x-k8s.io

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/e11d3a7a92224be5939dcc4ebd8adfcb

:x: e2e-openstack-quick FAILURE in 21m 29s

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/ad2876133f2240c7a4e13d35ad74dad9

:x: e2e-openstack-quick FAILURE in 31m 06s

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/16e058956fe3455687f879e9714a6a84

:x: e2e-openstack-quick FAILURE in 23m 26s

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/43f73e1da32e4bd189178ab9af29060d

:x: e2e-openstack-quick FAILURE in 26m 18s

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/c2dd26aeba734eb09a9d33360f65ce48

:x: e2e-openstack-quick FAILURE in 26m 34s

[scs-zuul[bot]]

Build canceled (e2e-quick-test pipeline).

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/0c7e05be0b4a4c3d93a5b307dbf8fb23

:x: e2e-openstack-quick FAILURE in 14m 37s

[michal-gubricky]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/0c7e05be0b4a4c3d93a5b307dbf8fb23

❌ e2e-openstack-quick FAILURE in 14m 37s

Current if only fails because of this error:

2024-07-10 11:48:56.617359 | ubuntu-jammy | E0710 11:44:49.489051       1 reconcile_state.go:1233] "Failed to create object" err="failed to create typed patch object (default/test-cluster-r25zc; infrastructure.cluster.x-k8s.io/v1beta1, Kind=OpenStackMachineTemplate): .spec.template.spec.serverGroupID: field not declared in schema" controller="topology/cluster" controllerGroup="cluster.x-k8s.io" controllerKind="Cluster" Cluster="default/test-cluster" namespace="default" name="test-cluster" reconcileID="5ecd0730-354a-499b-9976-3c7053b36f29" resource={"group":"infrastructure.cluster.x-k8s.io","version":"v1beta1","resource":"OpenStackMachineTemplate"} OpenStackMachineTemplate="default/test-cluster-r25zc" OpenStackMachineTemplate="default/test-cluster-r25zc"

but after merging the main into this branch, it should work.

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/f0c70acdbe0a477b88a966695704bde7

:x: e2e-openstack-quick FAILURE in 18m 17s

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/36f231c83e1f477e98658d82d4356632

:x: e2e-openstack-quick FAILURE in 21m 48s

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/101b6114d227423c9e2967659f6bf970

:x: e2e-openstack-quick FAILURE in 24m 21s

[michal-gubricky]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/36f231c83e1f477e98658d82d4356632

❌ e2e-openstack-quick FAILURE in 21m 48s

The pipeline failed on the sonobuoy tests, which is not related to this issue #133

sonobuoy run --plugin-env=e2e.E2E_PROVIDER=openstack --mode=quick
time="2024-07-10T14:07:15Z" level=error msg="Preflight checks failed"
time="2024-07-10T14:07:15Z" level=error msg="no dns pods found with the labels [k8s-app=kube-dns, k8s-app=coredns] in namespace kube-system"

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/999743e2cdb041c3a04d29286288c232

:x: e2e-openstack-quick FAILURE in 37m 19s

Warning:

SCS Compliance results

Testing SCS Compatible KaaS version v2 ******************************************************* Testing standard Kubernetes version policy ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0210-v2-k8s-version-policy.md ... INFO: Checking cluster specified by default context in /home/ubuntu/cluster_manifest/kubeconfig-test-cluster. INFO: The K8s cluster version 1.28.11 of cluster 'test-cluster-admin@test-cluster' is still in the recency time window. ... returned 0 errors, 0 aborts ******************************************************* Testing standard Kubernetes node distribution and availability ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0214-v1-k8s-node-distribution.md ... WARNING: There seems to be no distribution across multiple regions or labels aren't set correctly across nodes. WARNING: There seems to be no distribution across multiple zones or labels aren't set correctly across nodes. INFO: The control nodes are distributed across 3 host-ids. WARNING: There seems to be no distribution across multiple regions or labels aren't set correctly across nodes. WARNING: There seems to be no distribution across multiple zones or labels aren't set correctly across nodes. WARNING: There seems to be no distribution across multiple host-ids or labels aren't set correctly across nodes. WARNING: No node distribution could be detected for the worker nodes. This produces only a warning, since it is just a recommendation. ... returned 0 errors, 0 aborts ******************************************************* Testing standard CNCF Kubernetes conformance ... Reference: https://github.com/cncf/k8s-conformance/tree/master ... WARNING: No check tool specified for CNCF Kubernetes conformance ******************************************************* Verdict for subject KaaS_V2, SCS Compatible KaaS, version v2: PASSED Testing SCS Compatible KaaS version v1 ******************************************************* Testing standard Kubernetes version policy ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0210-v2-k8s-version-policy.md ... ... returned 0 errors, 0 aborts ******************************************************* Testing standard Kubernetes node distribution and availability ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0214-v1-k8s-node-distribution.md ... CRITICAL: [Errno 2] No such file or directory: '/home/ubuntu/scs-compliance/Tests/kaas/k8s-node-distribution/k8s-node-distribution-check.py' ... returned 0 errors, 1 aborts ******************************************************* Verdict for subject KaaS_V2, SCS Compatible KaaS, version v1: 1 ERRORS

Sonobouy results

Plugin: e2e Status: failed Total: 7394 Passed: 4 Failed: 1 Skipped: 7389 Failed tests: [sig-node] Pods should be submitted and removed [NodeConformance] [Conformance] Plugin: systemd-logs Status: failed Total: 5 Passed: 0 Failed: 5 Skipped: 0 Failed tests: No pod was scheduled on node test-cluster-bkncd-7kl62 within 27.252284022s. Check tolerations for plugin systemd-logs No pod was scheduled on node test-cluster-bkncd-wdpsz within 27.252409372s. Check tolerations for plugin systemd-logs No pod was scheduled on node test-cluster-openstack-scs-1-28-phghv-rr2kt-2g2wb within 27.252421903s. Check tolerations for plugin systemd-logs No pod was scheduled on node test-cluster-openstack-scs-1-28-phghv-rr2kt-pqzrg within 27.252432577s. Check tolerations for plugin systemd-logs No pod was scheduled on node test-cluster-openstack-scs-1-28-phghv-rr2kt-xns9g within 27.252437067s. Check tolerations for plugin systemd-logs Run Details: API Server version: v1.28.11 Node health: 2/6 (33%) Details for failed nodes: test-cluster-bkncd-2vrwf Ready:False: KubeletNotReady: [container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized, CSINode is not yet initialized] test-cluster-bkncd-wdpsz Ready:Unknown: NodeStatusUnknown: Kubelet stopped posting node status. test-cluster-openstack-scs-1-28-phghv-rr2kt-2g2wb Ready:Unknown: NodeStatusUnknown: Kubelet stopped posting node status. test-cluster-openstack-scs-1-28-phghv-rr2kt-xns9g Ready:Unknown: NodeStatusUnknown: Kubelet stopped posting node status. Pods health: 37/42 (88%) Details for failed pods: kube-system/cilium-jgs6t Ready:: : kube-system/kube-proxy-fwt22 Ready:: : kube-system/openstack-cinder-csi-nodeplugin-4kw9v Ready:: : sonobuoy/sonobuoy-e2e-job-a07d6af1a77c4f59 Ready:False: PodFailed: sonobuoy/sonobuoy-systemd-logs-daemon-set-b588ee6650b449e0-b9wkv Ready:: : Errors detected in files: Errors: 32 podlogs/kube-system/cilium-96xsl/logs/cilium-agent.txt 8 podlogs/sonobuoy/sonobuoy/logs/kube-sonobuoy.txt 8 podlogs/sonobuoy/sonobuoy-e2e-job-a07d6af1a77c4f59/logs/e2e.txt 6 podlogs/sonobuoy/sonobuoy-systemd-logs-daemon-set-b588ee6650b449e0-552pt/logs/sonobuoy-worker.txt 6 podlogs/kube-system/cilium-2hm2v/logs/cilium-agent.txt 3 podlogs/sonobuoy/sonobuoy-e2e-job-a07d6af1a77c4f59/logs/sonobuoy-worker.txt 1 podlogs/sonobuoy/sonobuoy-systemd-logs-daemon-set-b588ee6650b449e0-2vwjk/logs/sonobuoy-worker.txt Warnings: 15 podlogs/kube-system/cilium-96xsl/logs/cilium-agent.txt 3 podlogs/sonobuoy/sonobuoy/logs/kube-sonobuoy.txt 2 podlogs/kube-system/cilium-2hm2v/logs/cilium-agent.txt 1 podlogs/sonobuoy/sonobuoy-e2e-job-a07d6af1a77c4f59/logs/e2e.txt 1 podlogs/sonobuoy/sonobuoy-systemd-logs-daemon-set-b588ee6650b449e0-552pt/logs/sonobuoy-worker.txt

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/a57242de6c4d4f6cade0ddb707623a6c

:x: e2e-openstack-quick FAILURE in 42m 31s

Warning:

SCS Compliance results

Testing SCS Compatible KaaS version v2 ******************************************************* Testing standard Kubernetes version policy ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0210-v2-k8s-version-policy.md ... INFO: Checking cluster specified by default context in /home/ubuntu/cluster_manifest/kubeconfig-test-cluster. INFO: The K8s cluster version 1.28.11 of cluster 'test-cluster-admin@test-cluster' is still in the recency time window. ... returned 0 errors, 0 aborts ******************************************************* Testing standard Kubernetes node distribution and availability ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0214-v1-k8s-node-distribution.md ... WARNING: There seems to be no distribution across multiple regions or labels aren't set correctly across nodes. WARNING: There seems to be no distribution across multiple zones or labels aren't set correctly across nodes. INFO: The control nodes are distributed across 3 host-ids. WARNING: There seems to be no distribution across multiple regions or labels aren't set correctly across nodes. WARNING: There seems to be no distribution across multiple zones or labels aren't set correctly across nodes. INFO: The worker nodes are distributed across 2 host-ids. ... returned 0 errors, 0 aborts ******************************************************* Testing standard CNCF Kubernetes conformance ... Reference: https://github.com/cncf/k8s-conformance/tree/master ... WARNING: No check tool specified for CNCF Kubernetes conformance ******************************************************* Verdict for subject KaaS_V2, SCS Compatible KaaS, version v2: PASSED Testing SCS Compatible KaaS version v1 ******************************************************* Testing standard Kubernetes version policy ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0210-v2-k8s-version-policy.md ... ... returned 0 errors, 0 aborts ******************************************************* Testing standard Kubernetes node distribution and availability ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0214-v1-k8s-node-distribution.md ... CRITICAL: [Errno 2] No such file or directory: '/home/ubuntu/scs-compliance/Tests/kaas/k8s-node-distribution/k8s-node-distribution-check.py' ... returned 0 errors, 1 aborts ******************************************************* Verdict for subject KaaS_V2, SCS Compatible KaaS, version v1: 1 ERRORS

Sonobouy results

Plugin: systemd-logs Status: passed Total: 5 Passed: 5 Failed: 0 Skipped: 0 Plugin: e2e Status: passed Total: 7394 Passed: 5 Failed: 0 Skipped: 7389 Run Details: API Server version: v1.28.11 Node health: 6/6 (100%) Pods health: 47/47 (100%) Errors detected in files: Errors: 10091 podlogs/kube-system/kube-apiserver-test-cluster-lv5ts-dzt24/logs/kube-apiserver.txt 8399 podlogs/kube-system/kube-apiserver-test-cluster-lv5ts-hpr5m/logs/kube-apiserver.txt 243 podlogs/kube-system/etcd-test-cluster-lv5ts-dzt24/logs/etcd.txt 222 podlogs/kube-system/etcd-test-cluster-lv5ts-hpr5m/logs/etcd.txt 182 podlogs/kube-system/kube-scheduler-test-cluster-lv5ts-vfhrr/logs/kube-scheduler.txt 125 podlogs/kube-system/etcd-test-cluster-lv5ts-vfhrr/logs/etcd.txt 118 podlogs/kube-system/kube-apiserver-test-cluster-lv5ts-vfhrr/logs/kube-apiserver.txt 30 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-2s6qn/logs/csi-snapshotter.txt 10 podlogs/kube-system/kube-controller-manager-test-cluster-lv5ts-vfhrr/logs/kube-controller-manager.txt 8 podlogs/kube-system/metrics-server-666c6745d5-b7gw6/logs/metrics-server.txt 7 podlogs/kube-system/kube-controller-manager-test-cluster-lv5ts-dzt24/logs/kube-controller-manager.txt 6 podlogs/kube-system/cilium-8vtz8/logs/cilium-agent.txt 6 podlogs/kube-system/cilium-twcbr/logs/cilium-agent.txt 5 podlogs/kube-system/cilium-js84x/logs/cilium-agent.txt 5 podlogs/kube-system/cilium-nz8hl/logs/cilium-agent.txt 5 podlogs/kube-system/cilium-operator-8645b8bb4f-fdkwt/logs/cilium-operator.txt 5 podlogs/kube-system/cilium-mbfcz/logs/cilium-agent.txt 5 podlogs/kube-system/cilium-2c2sm/logs/cilium-agent.txt 2 podlogs/kube-system/kube-scheduler-test-cluster-lv5ts-dzt24/logs/kube-scheduler.txt 2 podlogs/kube-system/openstack-cloud-controller-manager-xmxrk/logs/openstack-cloud-controller-manager.txt 2 podlogs/sonobuoy/sonobuoy-e2e-job-d8ed276cf1e44791/logs/sonobuoy-worker.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-trhb8/logs/node-driver-registrar.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-gxvb6/logs/node-driver-registrar.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-6prbc/logs/node-driver-registrar.txt 1 podlogs/sonobuoy/sonobuoy/logs/kube-sonobuoy.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-c2t5z/logs/node-driver-registrar.txt 1 podlogs/sonobuoy/sonobuoy-e2e-job-d8ed276cf1e44791/logs/e2e.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-mshfs/logs/node-driver-registrar.txt 1 podlogs/kube-system/openstack-cloud-controller-manager-w2gxf/logs/openstack-cloud-controller-manager.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-qkbg8/logs/node-driver-registrar.txt Warnings: 2949 podlogs/kube-system/etcd-test-cluster-lv5ts-dzt24/logs/etcd.txt 1740 podlogs/kube-system/etcd-test-cluster-lv5ts-hpr5m/logs/etcd.txt 966 podlogs/kube-system/etcd-test-cluster-lv5ts-vfhrr/logs/etcd.txt 78 podlogs/kube-system/kube-scheduler-test-cluster-lv5ts-vfhrr/logs/kube-scheduler.txt 65 podlogs/kube-system/kube-apiserver-test-cluster-lv5ts-dzt24/logs/kube-apiserver.txt 57 podlogs/kube-system/kube-apiserver-test-cluster-lv5ts-hpr5m/logs/kube-apiserver.txt 32 podlogs/kube-system/kube-apiserver-test-cluster-lv5ts-vfhrr/logs/kube-apiserver.txt 15 podlogs/kube-system/openstack-cinder-csi-nodeplugin-6prbc/logs/node-driver-registrar.txt 15 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-2s6qn/logs/csi-snapshotter.txt 9 podlogs/kube-system/openstack-cinder-csi-nodeplugin-gxvb6/logs/node-driver-registrar.txt 9 podlogs/kube-system/openstack-cinder-csi-nodeplugin-6prbc/logs/liveness-probe.txt 6 podlogs/kube-system/cilium-2c2sm/logs/cilium-agent.txt 5 podlogs/sonobuoy/sonobuoy/logs/kube-sonobuoy.txt 5 podlogs/kube-system/openstack-cinder-csi-nodeplugin-trhb8/logs/node-driver-registrar.txt 5 podlogs/kube-system/openstack-cinder-csi-nodeplugin-c2t5z/logs/node-driver-registrar.txt 4 podlogs/kube-system/openstack-cinder-csi-nodeplugin-qkbg8/logs/node-driver-registrar.txt 4 podlogs/kube-system/openstack-cloud-controller-manager-w2gxf/logs/openstack-cloud-controller-manager.txt 3 podlogs/kube-system/openstack-cinder-csi-nodeplugin-mshfs/logs/node-driver-registrar.txt 3 podlogs/kube-system/openstack-cinder-csi-nodeplugin-qkbg8/logs/liveness-probe.txt 3 podlogs/kube-system/openstack-cinder-csi-nodeplugin-c2t5z/logs/liveness-probe.txt 3 podlogs/kube-system/openstack-cinder-csi-nodeplugin-trhb8/logs/liveness-probe.txt 2 podlogs/kube-system/openstack-cinder-csi-nodeplugin-mshfs/logs/liveness-probe.txt 2 podlogs/kube-system/cilium-twcbr/logs/cilium-agent.txt 2 podlogs/kube-system/cilium-8vtz8/logs/cilium-agent.txt 2 podlogs/kube-system/openstack-cinder-csi-nodeplugin-gxvb6/logs/liveness-probe.txt 1 podlogs/kube-system/openstack-cloud-controller-manager-xmxrk/logs/openstack-cloud-controller-manager.txt 1 podlogs/kube-system/cilium-mbfcz/logs/cilium-agent.txt 1 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-2s6qn/logs/csi-provisioner.txt 1 podlogs/kube-system/openstack-cloud-controller-manager-lcpw8/logs/openstack-cloud-controller-manager.txt 1 podlogs/sonobuoy/sonobuoy-e2e-job-d8ed276cf1e44791/logs/e2e.txt 1 podlogs/kube-system/cilium-operator-8645b8bb4f-fdkwt/logs/cilium-operator.txt 1 podlogs/kube-system/cilium-nz8hl/logs/cilium-agent.txt 1 podlogs/kube-system/cilium-js84x/logs/cilium-agent.txt

[scs-zuul[bot]]

Build succeeded (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/a27b45f09b1c4cb9bf68e9c03af2157d

:heavy_check_mark: e2e-openstack-quick SUCCESS in 45m 20s

Warning:

SCS Compliance results

Testing SCS Compatible KaaS version v2 ******************************************************* Testing standard Kubernetes version policy ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0210-v2-k8s-version-policy.md ... INFO: Checking cluster specified by default context in /home/ubuntu/cluster_manifest/kubeconfig-test-cluster. INFO: The K8s cluster version 1.28.11 of cluster 'test-cluster-admin@test-cluster' is still in the recency time window. ... returned 0 errors, 0 aborts ******************************************************* Testing standard Kubernetes node distribution and availability ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0214-v1-k8s-node-distribution.md ... WARNING: There seems to be no distribution across multiple regions or labels aren't set correctly across nodes. WARNING: There seems to be no distribution across multiple zones or labels aren't set correctly across nodes. INFO: The control nodes are distributed across 2 host-ids. WARNING: There seems to be no distribution across multiple regions or labels aren't set correctly across nodes. WARNING: There seems to be no distribution across multiple zones or labels aren't set correctly across nodes. INFO: The worker nodes are distributed across 2 host-ids. ... returned 0 errors, 0 aborts ******************************************************* Testing standard CNCF Kubernetes conformance ... Reference: https://github.com/cncf/k8s-conformance/tree/master ... WARNING: No check tool specified for CNCF Kubernetes conformance ******************************************************* Verdict for subject KaaS_V2, SCS Compatible KaaS, version v2: PASSED Testing SCS Compatible KaaS version v1 ******************************************************* Testing standard Kubernetes version policy ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0210-v2-k8s-version-policy.md ... ... returned 0 errors, 0 aborts ******************************************************* Testing standard Kubernetes node distribution and availability ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0214-v1-k8s-node-distribution.md ... ... returned 0 errors, 0 aborts ******************************************************* Verdict for subject KaaS_V2, SCS Compatible KaaS, version v1: PASSED

Sonobouy results

Plugin: e2e Status: passed Total: 7394 Passed: 5 Failed: 0 Skipped: 7389 Plugin: systemd-logs Status: passed Total: 4 Passed: 4 Failed: 0 Skipped: 0 Run Details: API Server version: v1.28.11 Node health: 4/4 (100%) Pods health: 34/34 (100%) Errors detected in files: Errors: 7519 podlogs/kube-system/kube-apiserver-test-cluster-ll7m7-czwlm/logs/kube-apiserver.txt 136 podlogs/kube-system/kube-scheduler-test-cluster-ll7m7-hls5k/logs/kube-scheduler.txt 99 podlogs/kube-system/etcd-test-cluster-ll7m7-czwlm/logs/etcd.txt 48 podlogs/kube-system/kube-proxy-czqrc/logs/kube-proxy.txt 47 podlogs/kube-system/etcd-test-cluster-ll7m7-hls5k/logs/etcd.txt 28 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-9t2rv/logs/csi-snapshotter.txt 21 podlogs/kube-system/kube-controller-manager-test-cluster-ll7m7-hls5k/logs/kube-controller-manager.txt 16 podlogs/kube-system/kube-apiserver-test-cluster-ll7m7-hls5k/logs/kube-apiserver.txt 5 podlogs/kube-system/cilium-g88jt/logs/cilium-agent.txt 5 podlogs/kube-system/cilium-rk9zj/logs/cilium-agent.txt 5 podlogs/kube-system/cilium-d8psw/logs/cilium-agent.txt 5 podlogs/kube-system/cilium-lqdpj/logs/cilium-agent.txt 2 podlogs/kube-system/openstack-cloud-controller-manager-fssg4/logs/openstack-cloud-controller-manager.txt 2 podlogs/kube-system/cilium-operator-8645b8bb4f-bpz7x/logs/cilium-operator.txt 1 podlogs/kube-system/kube-controller-manager-test-cluster-ll7m7-czwlm/logs/kube-controller-manager.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-lndbc/logs/node-driver-registrar.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-8hq4w/logs/node-driver-registrar.txt 1 podlogs/sonobuoy/sonobuoy-e2e-job-ad9eb3e818974768/logs/e2e.txt 1 podlogs/kube-system/cilium-operator-8645b8bb4f-687qk/logs/cilium-operator.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-cglcj/logs/node-driver-registrar.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-mdvhd/logs/node-driver-registrar.txt 1 podlogs/kube-system/kube-scheduler-test-cluster-ll7m7-czwlm/logs/kube-scheduler.txt Warnings: 1430 podlogs/kube-system/etcd-test-cluster-ll7m7-czwlm/logs/etcd.txt 1159 podlogs/kube-system/etcd-test-cluster-ll7m7-hls5k/logs/etcd.txt 63 podlogs/kube-system/kube-scheduler-test-cluster-ll7m7-hls5k/logs/kube-scheduler.txt 44 podlogs/kube-system/kube-apiserver-test-cluster-ll7m7-hls5k/logs/kube-apiserver.txt 38 podlogs/kube-system/kube-apiserver-test-cluster-ll7m7-czwlm/logs/kube-apiserver.txt 18 podlogs/kube-system/kube-proxy-czqrc/logs/kube-proxy.txt 16 podlogs/kube-system/openstack-cinder-csi-nodeplugin-lndbc/logs/node-driver-registrar.txt 14 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-9t2rv/logs/csi-snapshotter.txt 9 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-9t2rv/logs/csi-attacher.txt 7 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-9t2rv/logs/csi-provisioner.txt 6 podlogs/kube-system/openstack-cinder-csi-nodeplugin-lndbc/logs/liveness-probe.txt 6 podlogs/kube-system/cilium-d8psw/logs/cilium-agent.txt 4 podlogs/kube-system/openstack-cloud-controller-manager-fssg4/logs/openstack-cloud-controller-manager.txt 4 podlogs/kube-system/openstack-cinder-csi-nodeplugin-8hq4w/logs/node-driver-registrar.txt 3 podlogs/kube-system/openstack-cinder-csi-nodeplugin-8hq4w/logs/liveness-probe.txt 3 podlogs/kube-system/openstack-cinder-csi-nodeplugin-cglcj/logs/liveness-probe.txt 3 podlogs/kube-system/openstack-cinder-csi-nodeplugin-cglcj/logs/node-driver-registrar.txt 2 podlogs/kube-system/openstack-cinder-csi-nodeplugin-mdvhd/logs/liveness-probe.txt 2 podlogs/kube-system/openstack-cinder-csi-nodeplugin-mdvhd/logs/node-driver-registrar.txt 1 podlogs/kube-system/cilium-lqdpj/logs/cilium-agent.txt 1 podlogs/kube-system/cilium-operator-8645b8bb4f-bpz7x/logs/cilium-operator.txt 1 podlogs/kube-system/cilium-operator-8645b8bb4f-687qk/logs/cilium-operator.txt 1 podlogs/kube-system/kube-controller-manager-test-cluster-ll7m7-hls5k/logs/kube-controller-manager.txt 1 podlogs/kube-system/openstack-cloud-controller-manager-7vkp7/logs/openstack-cloud-controller-manager.txt 1 podlogs/sonobuoy/sonobuoy-e2e-job-ad9eb3e818974768/logs/e2e.txt 1 podlogs/kube-system/cilium-rk9zj/logs/cilium-agent.txt 1 podlogs/kube-system/cilium-g88jt/logs/cilium-agent.txt 1 podlogs/sonobuoy/sonobuoy/logs/kube-sonobuoy.txt

[scs-zuul[bot]]

Build failed (e2e-test pipeline). https://zuul.scs.community/t/SCS/buildset/e506c757a51d413aa411d168bc854407

:x: e2e-openstack-conformance FAILURE in 31m 20s

[scs-zuul[bot]]

Build succeeded (e2e-test pipeline). https://zuul.scs.community/t/SCS/buildset/ce536045c166419a811e0095d2c8e944

:heavy_check_mark: e2e-openstack-conformance SUCCESS in 42m 31s

Warning:

SCS Compliance results

Testing SCS Compatible KaaS version v2 ******************************************************* Testing standard Kubernetes version policy ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0210-v2-k8s-version-policy.md ... INFO: Checking cluster specified by default context in /home/ubuntu/cluster_manifest/kubeconfig-test-cluster. INFO: The K8s cluster version 1.28.11 of cluster 'test-cluster-admin@test-cluster' is still in the recency time window. ... returned 0 errors, 0 aborts ******************************************************* Testing standard Kubernetes node distribution and availability ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0214-v1-k8s-node-distribution.md ... WARNING: There seems to be no distribution across multiple regions or labels aren't set correctly across nodes. WARNING: There seems to be no distribution across multiple zones or labels aren't set correctly across nodes. INFO: The control nodes are distributed across 3 host-ids. WARNING: There seems to be no distribution across multiple regions or labels aren't set correctly across nodes. WARNING: There seems to be no distribution across multiple zones or labels aren't set correctly across nodes. INFO: The worker nodes are distributed across 3 host-ids. ... returned 0 errors, 0 aborts ******************************************************* Testing standard CNCF Kubernetes conformance ... Reference: https://github.com/cncf/k8s-conformance/tree/master ... WARNING: No check tool specified for CNCF Kubernetes conformance ******************************************************* Verdict for subject KaaS_V2, SCS Compatible KaaS, version v2: PASSED Testing SCS Compatible KaaS version v1 ******************************************************* Testing standard Kubernetes version policy ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0210-v2-k8s-version-policy.md ... ... returned 0 errors, 0 aborts ******************************************************* Testing standard Kubernetes node distribution and availability ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0214-v1-k8s-node-distribution.md ... ... returned 0 errors, 0 aborts ******************************************************* Verdict for subject KaaS_V2, SCS Compatible KaaS, version v1: PASSED

Sonobouy results

Plugin: e2e Status: failed Total: 4 Passed: 3 Failed: 1 Skipped: 0 Failed tests: [SynchronizedBeforeSuite] Plugin: systemd-logs Status: passed Total: 5 Passed: 5 Failed: 0 Skipped: 0 Run Details: API Server version: Node health: 0/0

[scs-zuul[bot]]

Build failed (e2e-test pipeline). https://zuul.scs.community/t/SCS/buildset/81e46b5d2adc4322b15f676617552efe

:x: e2e-openstack-conformance FAILURE in 38m 25s

[scs-zuul[bot]]

Build succeeded (e2e-test pipeline). https://zuul.scs.community/t/SCS/buildset/dd56126bb5204076914fa4acf3a5b21d

:heavy_check_mark: e2e-openstack-conformance SUCCESS in 2h 49m 46s

Warning:

SCS Compliance results

Testing SCS Compatible KaaS version v2 ******************************************************* Testing standard Kubernetes version policy ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0210-v2-k8s-version-policy.md ... INFO: Checking cluster specified by default context in /home/ubuntu/cluster_manifest/kubeconfig-test-cluster. INFO: The K8s cluster version 1.28.11 of cluster 'test-cluster-admin@test-cluster' is still in the recency time window. ... returned 0 errors, 0 aborts ******************************************************* Testing standard Kubernetes node distribution and availability ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0214-v1-k8s-node-distribution.md ... WARNING: There seems to be no distribution across multiple regions or labels aren't set correctly across nodes. WARNING: There seems to be no distribution across multiple zones or labels aren't set correctly across nodes. INFO: The control nodes are distributed across 3 host-ids. WARNING: There seems to be no distribution across multiple regions or labels aren't set correctly across nodes. WARNING: There seems to be no distribution across multiple zones or labels aren't set correctly across nodes. INFO: The worker nodes are distributed across 2 host-ids. ... returned 0 errors, 0 aborts ******************************************************* Testing standard CNCF Kubernetes conformance ... Reference: https://github.com/cncf/k8s-conformance/tree/master ... WARNING: No check tool specified for CNCF Kubernetes conformance ******************************************************* Verdict for subject KaaS_V2, SCS Compatible KaaS, version v2: PASSED Testing SCS Compatible KaaS version v1 ******************************************************* Testing standard Kubernetes version policy ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0210-v2-k8s-version-policy.md ... ... returned 0 errors, 0 aborts ******************************************************* Testing standard Kubernetes node distribution and availability ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0214-v1-k8s-node-distribution.md ... ... returned 0 errors, 0 aborts ******************************************************* Verdict for subject KaaS_V2, SCS Compatible KaaS, version v1: PASSED

Sonobouy results

Plugin: e2e Status: failed Total: 7394 Passed: 381 Failed: 3 Skipped: 7010 Failed tests: [sig-network] Services should have session affinity work for service with type clusterIP [LinuxOnly] [Conformance] [sig-network] HostPort validates that there is no conflict between pods with same hostPort but different hostIP and protocol [LinuxOnly] [Conformance] [sig-network] Services should be able to switch session affinity for service with type clusterIP [LinuxOnly] [Conformance] Plugin: systemd-logs Status: passed Total: 5 Passed: 5 Failed: 0 Skipped: 0 Run Details: API Server version: v1.28.11 Node health: 6/6 (100%) Pods health: 46/47 (97%) Details for failed pods: sonobuoy/sonobuoy-e2e-job-0904cd9799214a07 Ready:False: PodFailed: Errors detected in files: Errors: 1489 podlogs/kube-system/etcd-test-cluster-t57bh-krsgw/logs/etcd.txt 597 podlogs/kube-system/etcd-test-cluster-t57bh-lbw7s/logs/etcd.txt 426 podlogs/kube-system/cilium-pb7nm/logs/cilium-agent.txt 372 podlogs/kube-system/cilium-4sv7w/logs/cilium-agent.txt 334 podlogs/kube-system/etcd-test-cluster-t57bh-9pts2/logs/etcd.txt 329 podlogs/kube-system/kube-apiserver-test-cluster-t57bh-9pts2/logs/kube-apiserver.txt 300 podlogs/kube-system/kube-apiserver-test-cluster-t57bh-lbw7s/logs/kube-apiserver.txt 295 podlogs/kube-system/cilium-r82f2/logs/cilium-agent.txt 240 podlogs/kube-system/kube-scheduler-test-cluster-t57bh-9pts2/logs/kube-scheduler.txt 204 podlogs/sonobuoy/sonobuoy-e2e-job-0904cd9799214a07/logs/e2e.txt 183 podlogs/kube-system/cilium-hb79v/logs/cilium-agent.txt 178 podlogs/kube-system/kube-controller-manager-test-cluster-t57bh-krsgw/logs/kube-controller-manager.txt 172 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-bxz4x/logs/csi-snapshotter.txt 143 podlogs/kube-system/kube-apiserver-test-cluster-t57bh-krsgw/logs/kube-apiserver.txt 100 podlogs/kube-system/cilium-wx7hm/logs/cilium-agent.txt 54 podlogs/kube-system/kube-proxy-cwq9c/logs/kube-proxy.txt 44 podlogs/kube-system/kube-proxy-bzrtj/logs/kube-proxy.txt 39 podlogs/sonobuoy/sonobuoy/logs/kube-sonobuoy.txt 37 podlogs/kube-system/kube-controller-manager-test-cluster-t57bh-9pts2/logs/kube-controller-manager.txt 35 podlogs/kube-system/kube-proxy-bxfdc/logs/kube-proxy.txt 35 podlogs/kube-system/kube-proxy-xf8dz/logs/kube-proxy.txt 32 podlogs/kube-system/coredns-5dd5756b68-8fl9z/logs/coredns.txt 25 podlogs/kube-system/coredns-5dd5756b68-vn7ww/logs/coredns.txt 8 podlogs/kube-system/kube-proxy-l9snp/logs/kube-proxy.txt 8 podlogs/kube-system/cilium-tw26g/logs/cilium-agent.txt 8 podlogs/kube-system/kube-proxy-j72pq/logs/kube-proxy.txt 4 podlogs/kube-system/openstack-cloud-controller-manager-x6glz/logs/openstack-cloud-controller-manager.txt 3 podlogs/sonobuoy/sonobuoy-systemd-logs-daemon-set-fe5af3d5ff474ad4-t4frc/logs/sonobuoy-worker.txt 3 podlogs/kube-system/cilium-operator-8645b8bb4f-hvzk2/logs/cilium-operator.txt 2 podlogs/kube-system/openstack-cloud-controller-manager-wv75c/logs/openstack-cloud-controller-manager.txt 2 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-bxz4x/logs/liveness-probe.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-rmrqr/logs/node-driver-registrar.txt 1 podlogs/kube-system/openstack-cloud-controller-manager-c82rv/logs/openstack-cloud-controller-manager.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-gtzp9/logs/node-driver-registrar.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-cjdfg/logs/node-driver-registrar.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-qblpm/logs/node-driver-registrar.txt 1 podlogs/kube-system/kube-controller-manager-test-cluster-t57bh-lbw7s/logs/kube-controller-manager.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-r72w9/logs/node-driver-registrar.txt 1 podlogs/kube-system/kube-scheduler-test-cluster-t57bh-lbw7s/logs/kube-scheduler.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-9z4qj/logs/node-driver-registrar.txt Warnings: 10070 podlogs/kube-system/etcd-test-cluster-t57bh-krsgw/logs/etcd.txt 8767 podlogs/kube-system/etcd-test-cluster-t57bh-lbw7s/logs/etcd.txt 5152 podlogs/kube-system/etcd-test-cluster-t57bh-9pts2/logs/etcd.txt 2146 podlogs/kube-system/cilium-pb7nm/logs/cilium-agent.txt 910 podlogs/kube-system/cilium-4sv7w/logs/cilium-agent.txt 739 podlogs/kube-system/cilium-r82f2/logs/cilium-agent.txt 99 podlogs/kube-system/kube-apiserver-test-cluster-t57bh-krsgw/logs/kube-apiserver.txt 97 podlogs/kube-system/kube-apiserver-test-cluster-t57bh-lbw7s/logs/kube-apiserver.txt 96 podlogs/kube-system/kube-scheduler-test-cluster-t57bh-9pts2/logs/kube-scheduler.txt 87 podlogs/kube-system/cilium-hb79v/logs/cilium-agent.txt 86 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-bxz4x/logs/csi-snapshotter.txt 80 podlogs/kube-system/kube-apiserver-test-cluster-t57bh-9pts2/logs/kube-apiserver.txt 75 podlogs/kube-system/kube-controller-manager-test-cluster-t57bh-krsgw/logs/kube-controller-manager.txt 53 podlogs/kube-system/cilium-wx7hm/logs/cilium-agent.txt 26 podlogs/kube-system/openstack-cinder-csi-nodeplugin-qblpm/logs/node-driver-registrar.txt 24 podlogs/kube-system/openstack-cinder-csi-nodeplugin-qblpm/logs/liveness-probe.txt 20 podlogs/kube-system/kube-proxy-cwq9c/logs/kube-proxy.txt 18 podlogs/kube-system/kube-proxy-bzrtj/logs/kube-proxy.txt 15 podlogs/kube-system/openstack-cinder-csi-nodeplugin-r72w9/logs/node-driver-registrar.txt 12 podlogs/kube-system/cilium-tw26g/logs/cilium-agent.txt 12 podlogs/kube-system/kube-proxy-bxfdc/logs/kube-proxy.txt 12 podlogs/kube-system/openstack-cinder-csi-nodeplugin-9z4qj/logs/node-driver-registrar.txt 11 podlogs/kube-system/openstack-cinder-csi-nodeplugin-9z4qj/logs/liveness-probe.txt 10 podlogs/kube-system/kube-proxy-xf8dz/logs/kube-proxy.txt 9 podlogs/sonobuoy/sonobuoy/logs/kube-sonobuoy.txt 7 podlogs/kube-system/openstack-cinder-csi-nodeplugin-gtzp9/logs/node-driver-registrar.txt 7 podlogs/kube-system/openstack-cinder-csi-nodeplugin-cjdfg/logs/node-driver-registrar.txt 5 podlogs/kube-system/openstack-cinder-csi-nodeplugin-cjdfg/logs/liveness-probe.txt 5 podlogs/kube-system/openstack-cinder-csi-nodeplugin-gtzp9/logs/liveness-probe.txt 4 podlogs/kube-system/openstack-cinder-csi-nodeplugin-r72w9/logs/liveness-probe.txt 4 podlogs/kube-system/openstack-cinder-csi-nodeplugin-rmrqr/logs/node-driver-registrar.txt 4 podlogs/kube-system/openstack-cloud-controller-manager-x6glz/logs/openstack-cloud-controller-manager.txt 2 podlogs/kube-system/openstack-cinder-csi-nodeplugin-rmrqr/logs/liveness-probe.txt 1 podlogs/sonobuoy/sonobuoy-e2e-job-0904cd9799214a07/logs/e2e.txt 1 podlogs/kube-system/cilium-operator-8645b8bb4f-hvzk2/logs/cilium-operator.txt 1 podlogs/kube-system/openstack-cloud-controller-manager-wv75c/logs/openstack-cloud-controller-manager.txt 1 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-bxz4x/logs/csi-provisioner.txt 1 podlogs/kube-system/openstack-cloud-controller-manager-c82rv/logs/openstack-cloud-controller-manager.txt 1 podlogs/sonobuoy/sonobuoy-systemd-logs-daemon-set-fe5af3d5ff474ad4-t4frc/logs/sonobuoy-worker.txt

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/01f3303d8fa34b76924b3d403c2dc147

:x: e2e-openstack-quick FAILURE in 43m 12s

Warning:

SCS Compliance results

Testing SCS Compatible KaaS version v2 ******************************************************* Testing standard Kubernetes version policy ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0210-v2-k8s-version-policy.md ... INFO: Checking cluster specified by default context in /home/ubuntu/cluster_manifest/kubeconfig-test-cluster. INFO: The K8s cluster version 1.28.11 of cluster 'test-cluster-admin@test-cluster' is still in the recency time window. ... returned 0 errors, 0 aborts ******************************************************* Testing standard Kubernetes node distribution and availability ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0214-v1-k8s-node-distribution.md ... WARNING: There seems to be no distribution across multiple regions or labels aren't set correctly across nodes. WARNING: There seems to be no distribution across multiple zones or labels aren't set correctly across nodes. INFO: The control nodes are distributed across 3 host-ids. WARNING: There seems to be no distribution across multiple regions or labels aren't set correctly across nodes. WARNING: There seems to be no distribution across multiple zones or labels aren't set correctly across nodes. INFO: The worker nodes are distributed across 2 host-ids. ... returned 0 errors, 0 aborts ******************************************************* Testing standard CNCF Kubernetes conformance ... Reference: https://github.com/cncf/k8s-conformance/tree/master ... WARNING: No check tool specified for CNCF Kubernetes conformance ******************************************************* Verdict for subject KaaS_V2, SCS Compatible KaaS, version v2: PASSED Testing SCS Compatible KaaS version v1 ******************************************************* Testing standard Kubernetes version policy ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0210-v2-k8s-version-policy.md ... ... returned 0 errors, 0 aborts ******************************************************* Testing standard Kubernetes node distribution and availability ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0214-v1-k8s-node-distribution.md ... ... returned 0 errors, 0 aborts ******************************************************* Verdict for subject KaaS_V2, SCS Compatible KaaS, version v1: PASSED

Sonobouy results

Plugin: e2e Status: passed Total: 7394 Passed: 5 Failed: 0 Skipped: 7389 Plugin: systemd-logs Status: passed Total: 5 Passed: 5 Failed: 0 Skipped: 0 Run Details: API Server version: v1.28.11 Node health: 6/6 (100%) Pods health: 47/47 (100%) Errors detected in files: Errors: 24010 podlogs/kube-system/kube-apiserver-test-cluster-49xfv-2xg2r/logs/kube-apiserver.txt 256 podlogs/kube-system/kube-scheduler-test-cluster-49xfv-zt847/logs/kube-scheduler.txt 202 podlogs/kube-system/etcd-test-cluster-49xfv-2xg2r/logs/etcd.txt 161 podlogs/kube-system/kube-scheduler-test-cluster-49xfv-lhm55/logs/kube-scheduler.txt 151 podlogs/kube-system/etcd-test-cluster-49xfv-zt847/logs/etcd.txt 122 podlogs/kube-system/kube-apiserver-test-cluster-49xfv-zt847/logs/kube-apiserver.txt 111 podlogs/kube-system/etcd-test-cluster-49xfv-lhm55/logs/etcd.txt 82 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-rhg4k/logs/csi-snapshotter.txt 48 podlogs/kube-system/kube-apiserver-test-cluster-49xfv-lhm55/logs/kube-apiserver.txt 28 podlogs/kube-system/kube-controller-manager-test-cluster-49xfv-zt847/logs/kube-controller-manager.txt 26 podlogs/kube-system/kube-controller-manager-test-cluster-49xfv-2xg2r/logs/kube-controller-manager.txt 11 podlogs/kube-system/kube-controller-manager-test-cluster-49xfv-lhm55/logs/kube-controller-manager.txt 9 podlogs/kube-system/metrics-server-666c6745d5-lbqs8/logs/metrics-server.txt 6 podlogs/kube-system/cilium-operator-8645b8bb4f-mdhmk/logs/cilium-operator.txt 6 podlogs/sonobuoy/sonobuoy-systemd-logs-daemon-set-4ae2ef20b4744f0a-thp99/logs/sonobuoy-worker.txt 5 podlogs/kube-system/cilium-6xm72/logs/cilium-agent.txt 5 podlogs/kube-system/cilium-sl7r6/logs/cilium-agent.txt 5 podlogs/kube-system/cilium-p2xl7/logs/cilium-agent.txt 5 podlogs/kube-system/cilium-mjc5g/logs/cilium-agent.txt 5 podlogs/kube-system/cilium-4294f/logs/cilium-agent.txt 5 podlogs/kube-system/cilium-2fr2k/logs/cilium-agent.txt 3 podlogs/kube-system/kube-proxy-4pfnn/logs/kube-proxy.txt 3 podlogs/kube-system/openstack-cloud-controller-manager-klth5/logs/openstack-cloud-controller-manager.txt 3 podlogs/kube-system/cilium-operator-8645b8bb4f-46jcx/logs/cilium-operator.txt 3 podlogs/sonobuoy/sonobuoy/logs/kube-sonobuoy.txt 2 podlogs/kube-system/kube-scheduler-test-cluster-49xfv-2xg2r/logs/kube-scheduler.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-vw96v/logs/node-driver-registrar.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-hcfpp/logs/node-driver-registrar.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-24xzv/logs/node-driver-registrar.txt 1 podlogs/sonobuoy/sonobuoy-e2e-job-91762fe8fcb94bee/logs/e2e.txt 1 podlogs/kube-system/openstack-cloud-controller-manager-vqr4j/logs/openstack-cloud-controller-manager.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-cff6s/logs/node-driver-registrar.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-5gldw/logs/node-driver-registrar.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-qcx5b/logs/node-driver-registrar.txt Warnings: 4627 podlogs/kube-system/etcd-test-cluster-49xfv-2xg2r/logs/etcd.txt 2734 podlogs/kube-system/etcd-test-cluster-49xfv-lhm55/logs/etcd.txt 2234 podlogs/kube-system/etcd-test-cluster-49xfv-zt847/logs/etcd.txt 111 podlogs/kube-system/kube-scheduler-test-cluster-49xfv-zt847/logs/kube-scheduler.txt 81 podlogs/kube-system/kube-scheduler-test-cluster-49xfv-lhm55/logs/kube-scheduler.txt 57 podlogs/kube-system/kube-apiserver-test-cluster-49xfv-2xg2r/logs/kube-apiserver.txt 56 podlogs/kube-system/kube-apiserver-test-cluster-49xfv-lhm55/logs/kube-apiserver.txt 42 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-rhg4k/logs/csi-snapshotter.txt 34 podlogs/kube-system/kube-apiserver-test-cluster-49xfv-zt847/logs/kube-apiserver.txt 26 podlogs/kube-system/openstack-cinder-csi-nodeplugin-vw96v/logs/node-driver-registrar.txt 21 podlogs/kube-system/kube-controller-manager-test-cluster-49xfv-2xg2r/logs/kube-controller-manager.txt 14 podlogs/kube-system/openstack-cinder-csi-nodeplugin-5gldw/logs/node-driver-registrar.txt 12 podlogs/kube-system/openstack-cinder-csi-nodeplugin-cff6s/logs/node-driver-registrar.txt 12 podlogs/kube-system/openstack-cinder-csi-nodeplugin-vw96v/logs/liveness-probe.txt 10 podlogs/kube-system/openstack-cinder-csi-nodeplugin-24xzv/logs/node-driver-registrar.txt 6 podlogs/kube-system/cilium-mjc5g/logs/cilium-agent.txt 4 podlogs/kube-system/openstack-cinder-csi-nodeplugin-cff6s/logs/liveness-probe.txt 4 podlogs/kube-system/openstack-cinder-csi-nodeplugin-hcfpp/logs/node-driver-registrar.txt 4 podlogs/kube-system/openstack-cloud-controller-manager-klth5/logs/openstack-cloud-controller-manager.txt 4 podlogs/kube-system/openstack-cinder-csi-nodeplugin-5gldw/logs/liveness-probe.txt 3 podlogs/kube-system/openstack-cinder-csi-nodeplugin-qcx5b/logs/node-driver-registrar.txt 3 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-rhg4k/logs/csi-attacher.txt 3 podlogs/kube-system/kube-scheduler-test-cluster-49xfv-2xg2r/logs/kube-scheduler.txt 3 podlogs/kube-system/openstack-cinder-csi-nodeplugin-hcfpp/logs/liveness-probe.txt 2 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-rhg4k/logs/csi-provisioner.txt 2 podlogs/kube-system/openstack-cinder-csi-nodeplugin-qcx5b/logs/liveness-probe.txt 2 podlogs/kube-system/openstack-cinder-csi-nodeplugin-24xzv/logs/liveness-probe.txt 1 podlogs/sonobuoy/sonobuoy-systemd-logs-daemon-set-4ae2ef20b4744f0a-thp99/logs/sonobuoy-worker.txt 1 podlogs/kube-system/cilium-4294f/logs/cilium-agent.txt 1 podlogs/kube-system/cilium-sl7r6/logs/cilium-agent.txt 1 podlogs/kube-system/openstack-cloud-controller-manager-vqr4j/logs/openstack-cloud-controller-manager.txt 1 podlogs/sonobuoy/sonobuoy-e2e-job-91762fe8fcb94bee/logs/e2e.txt 1 podlogs/kube-system/cilium-operator-8645b8bb4f-mdhmk/logs/cilium-operator.txt 1 podlogs/kube-system/openstack-cloud-controller-manager-p42w7/logs/openstack-cloud-controller-manager.txt 1 podlogs/sonobuoy/sonobuoy/logs/kube-sonobuoy.txt 1 podlogs/kube-system/cilium-2fr2k/logs/cilium-agent.txt 1 podlogs/kube-system/cilium-p2xl7/logs/cilium-agent.txt 1 podlogs/kube-system/cilium-operator-8645b8bb4f-46jcx/logs/cilium-operator.txt 1 podlogs/kube-system/cilium-6xm72/logs/cilium-agent.txt

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/91e9305018cb4d7bb30fca0a7eee3d02

:x: e2e-openstack-quick FAILURE in 14m 41s

[scs-zuul[bot]]

Build failed (e2e-test pipeline). https://zuul.scs.community/t/SCS/buildset/a70b33678cf842c78da8f044ad92185c

:x: e2e-openstack-conformance FAILURE in 18m 55s

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/e651cdb2415f42299ec522fbd8e59583

:x: e2e-openstack-quick FAILURE in 15m 34s

[michal-gubricky]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/e651cdb2415f42299ec522fbd8e59583

❌ e2e-openstack-quick FAILURE in 15m 34s

The template cluster.yaml.j2 needs to be changed as machineDeployments were renamed, see https://github.com/SovereignCloudStack/cluster-stacks/commit/3e7c580595fd3899d85655c744df72ac6c252462

[chess-knight]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/e651cdb2415f42299ec522fbd8e59583 ❌ e2e-openstack-quick FAILURE in 15m 34s

The template cluster.yaml.j2 needs to be changed as machineDeployments were renamed, see 3e7c580

I merged changes after a failed build. It looks like we have the same problem as in KaaSv1 and should be fixed also here, see https://github.com/SovereignCloudStack/k8s-cluster-api-provider/pull/621

[scs-zuul[bot]]

Build failed (e2e-quick-test pipeline). https://zuul.scs.community/t/SCS/buildset/a1ec74f6057f4b019645654b9e6dbd3e

:x: e2e-openstack-quick FAILURE in 34m 06s

Warning:

SCS Compliance results

Testing SCS Compatible KaaS version v2 ******************************************************* Testing standard Kubernetes version policy ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0210-v2-k8s-version-policy.md ... INFO: Checking cluster specified by default context in /home/ubuntu/cluster_manifest/kubeconfig-test-cluster. INFO: The K8s cluster version 1.28.11 of cluster 'test-cluster-admin@test-cluster' is still in the recency time window. ... returned 0 errors, 0 aborts ******************************************************* Testing standard Kubernetes node distribution and availability ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0214-v1-k8s-node-distribution.md ... WARNING: There seems to be no distribution across multiple regions or labels aren't set correctly across nodes. WARNING: There seems to be no distribution across multiple zones or labels aren't set correctly across nodes. WARNING: There seems to be no distribution across multiple host-ids or labels aren't set correctly across nodes. ERROR: The distribution of nodes described in the standard couldn't be detected. ... returned 1 errors, 0 aborts ******************************************************* Testing standard CNCF Kubernetes conformance ... Reference: https://github.com/cncf/k8s-conformance/tree/master ... WARNING: No check tool specified for CNCF Kubernetes conformance ******************************************************* Verdict for subject KaaS_V2, SCS Compatible KaaS, version v2: 1 ERRORS Testing SCS Compatible KaaS version v1 ******************************************************* Testing standard Kubernetes version policy ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0210-v2-k8s-version-policy.md ... ... returned 0 errors, 0 aborts ******************************************************* Testing standard Kubernetes node distribution and availability ... Reference: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0214-v1-k8s-node-distribution.md ... ... returned 1 errors, 0 aborts ******************************************************* Verdict for subject KaaS_V2, SCS Compatible KaaS, version v1: 1 ERRORS

Sonobouy results

Plugin: e2e Status: passed Total: 7394 Passed: 5 Failed: 0 Skipped: 7389 Plugin: systemd-logs Status: passed Total: 4 Passed: 4 Failed: 0 Skipped: 0 Run Details: API Server version: v1.28.11 Node health: 4/4 (100%) Pods health: 29/29 (100%) Errors detected in files: Errors: 227 podlogs/kube-system/kube-apiserver-test-cluster-n6jgv-dcdk7/logs/kube-apiserver.txt 79 podlogs/kube-system/kube-scheduler-test-cluster-n6jgv-dcdk7/logs/kube-scheduler.txt 39 podlogs/kube-system/kube-proxy-gsx9v/logs/kube-proxy.txt 36 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-bvznh/logs/csi-snapshotter.txt 30 podlogs/kube-system/kube-proxy-j2lwd/logs/kube-proxy.txt 23 podlogs/kube-system/kube-controller-manager-test-cluster-n6jgv-dcdk7/logs/kube-controller-manager.txt 6 podlogs/kube-system/cilium-8drfc/logs/cilium-agent.txt 5 podlogs/kube-system/cilium-wbvg7/logs/cilium-agent.txt 5 podlogs/kube-system/cilium-mls4p/logs/cilium-agent.txt 5 podlogs/kube-system/cilium-zgdlg/logs/cilium-agent.txt 2 podlogs/kube-system/etcd-test-cluster-n6jgv-dcdk7/logs/etcd.txt 2 podlogs/kube-system/openstack-cloud-controller-manager-ncvl9/logs/openstack-cloud-controller-manager.txt 2 podlogs/kube-system/cilium-operator-8645b8bb4f-d5m4z/logs/cilium-operator.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-5lb2k/logs/node-driver-registrar.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-dzwnh/logs/node-driver-registrar.txt 1 podlogs/kube-system/cilium-operator-8645b8bb4f-k4dwc/logs/cilium-operator.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-79bb6/logs/node-driver-registrar.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-vxd55/logs/node-driver-registrar.txt 1 podlogs/sonobuoy/sonobuoy-e2e-job-b2843dd5383e4332/logs/e2e.txt Warnings: 838 podlogs/kube-system/etcd-test-cluster-n6jgv-dcdk7/logs/etcd.txt 43 podlogs/kube-system/kube-apiserver-test-cluster-n6jgv-dcdk7/logs/kube-apiserver.txt 42 podlogs/kube-system/kube-scheduler-test-cluster-n6jgv-dcdk7/logs/kube-scheduler.txt 18 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-bvznh/logs/csi-snapshotter.txt 15 podlogs/kube-system/kube-proxy-gsx9v/logs/kube-proxy.txt 12 podlogs/kube-system/kube-proxy-j2lwd/logs/kube-proxy.txt 10 podlogs/kube-system/openstack-cinder-csi-nodeplugin-dzwnh/logs/node-driver-registrar.txt 6 podlogs/kube-system/openstack-cinder-csi-nodeplugin-vxd55/logs/node-driver-registrar.txt 6 podlogs/kube-system/cilium-zgdlg/logs/cilium-agent.txt 4 podlogs/kube-system/openstack-cloud-controller-manager-ncvl9/logs/openstack-cloud-controller-manager.txt 4 podlogs/kube-system/openstack-cinder-csi-nodeplugin-vxd55/logs/liveness-probe.txt 4 podlogs/kube-system/openstack-cinder-csi-nodeplugin-dzwnh/logs/liveness-probe.txt 2 podlogs/kube-system/openstack-cinder-csi-nodeplugin-5lb2k/logs/node-driver-registrar.txt 2 podlogs/kube-system/cilium-8drfc/logs/cilium-agent.txt 2 podlogs/kube-system/kube-controller-manager-test-cluster-n6jgv-dcdk7/logs/kube-controller-manager.txt 2 podlogs/kube-system/openstack-cinder-csi-nodeplugin-5lb2k/logs/liveness-probe.txt 1 podlogs/sonobuoy/sonobuoy-e2e-job-b2843dd5383e4332/logs/e2e.txt 1 podlogs/sonobuoy/sonobuoy/logs/kube-sonobuoy.txt 1 podlogs/kube-system/cilium-operator-8645b8bb4f-d5m4z/logs/cilium-operator.txt 1 podlogs/kube-system/cilium-wbvg7/logs/cilium-agent.txt 1 podlogs/kube-system/openstack-cinder-csi-nodeplugin-79bb6/logs/liveness-probe.txt 1 podlogs/kube-system/cilium-operator-8645b8bb4f-k4dwc/logs/cilium-operator.txt 1 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-bvznh/logs/csi-attacher.txt 1 podlogs/kube-system/cilium-mls4p/logs/cilium-agent.txt 1 podlogs/kube-system/openstack-cinder-csi-controllerplugin-78c4557887-bvznh/logs/csi-provisioner.txt