kubernetes-sigs / security-profiles-operator

The Kubernetes Security Profiles Operator
Apache License 2.0
697 stars 107 forks source link

Running e2e tests on OpenShift hang #1976

Closed rhmdnd closed 8 months ago

rhmdnd commented 10 months ago

What happened:

Running e2e tests using an OpenShift cluster failed because the spods were caught in a crash loop backoff due to using the wrong image.

What you expected to happen:

I expected the e2e test suite to run normally.

How to reproduce it (as minimally and precisely as possible):

Follow the hacking guide for running tests on an OpenShift cluster.

https://github.com/kubernetes-sigs/security-profiles-operator/blob/main/hacking.md#distribution-specific-instructions-openshift-1

First I pushed the images to the OpenShift registry using:

make push-openshift-dev

Then, I started the e2e tests using the documented variables:

$ E2E_SPO_IMAGE=image-registry.openshift-image-registry.svc:5000/openshift/security-profiles-operator:latest \ 
E2E_CLUSTER_TYPE=openshift \
E2E_SKIP_BUILD_IMAGES=true \
E2E_TEST_SECCOMP=false \
E2E_TEST_BPF_RECORDER=false \
E2E_TEST_LOG_ENRICHER=false \
make test-e2e

The operator container seems to come up fine, but the spod containers get stuck in a crash loop.

Anything else we need to know?:

Diffing the repository I noticed the following changes were made to the deployment, prior to the install:

diff --git a/deploy/operator.yaml b/deploy/operator.yaml
index 2967d3d9..4c96d640 100644
--- a/deploy/operator.yaml
+++ b/deploy/operator.yaml
@@ -3190,9 +3190,9 @@ spec:
         - name: RELATED_IMAGE_SELINUXD
           value: quay.io/security-profiles-operator/selinuxd
         - name: RELATED_IMAGE_SELINUXD_EL8
-          value: quay.io/security-profiles-operator/selinuxd-el8:latest
+          value: quay.io/security-profiles-operator/selinuxd
         - name: RELATED_IMAGE_SELINUXD_EL9
-          value: quay.io/security-profiles-operator/selinuxd-el9:latest
+          value: quay.io/security-profiles-operator/selinuxd
         - name: OPERATOR_NAMESPACE
           valueFrom:
             fieldRef:
@@ -3208,7 +3208,7 @@ spec:
               fieldPath: spec.nodeName
         - name: KUBELET_DIR
           value: /var/lib/kubelet
-        image: gcr.io/k8s-staging-sp-operator/security-profiles-operator:latest
+        image: image-registry.openshift-image-registry.svc:5000/openshift/security-profiles-operator:latest
         imagePullPolicy: Always
         name: security-profiles-operator
         resources:

I manually reverted the change to point the EL8 and EL9 image locations back to selinuxd-el8 and selinuxd-el9, respectively, After I did that, the spods came up fine.

Environment:

saschagrunert commented 10 months ago

Hm, we can probably stick to the el8 and el9 images for the tests then. @jhrozek any thoughts on that?