openshift / file-integrity-operator

Operator providing OpenShift cluster node file integrity checking
Apache License 2.0
31 stars 27 forks source link

E2E testing fails due to failed node taints #439

Closed rhmdnd closed 9 months ago

rhmdnd commented 1 year ago

We're experiencing an issue in CI where the E2E testing fails because a node can't be tainted, which manifests in the follow error:

 === RUN   TestFileIntegrityTolerations
&Namespace{ObjectMeta:{osdk-e2e-6c36263d-255d-4a22-9659-ba12e7965054      0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[pod-security.kubernetes.io/enforce:privileged security.openshift.io/scc.podSecurityLabelSync:false] map[] [] [] []},Spec:NamespaceSpec{Finalizers:[],},Status:NamespaceStatus{Phase:,Conditions:[]NamespaceCondition{},},}
    client.go:47: resource type ServiceAccount with namespace/name (osdk-e2e-6c36263d-255d-4a22-9659-ba12e7965054/file-integrity-daemon) created
    client.go:47: resource type ServiceAccount with namespace/name (osdk-e2e-6c36263d-255d-4a22-9659-ba12e7965054/file-integrity-operator) created
    client.go:47: resource type Role with namespace/name (osdk-e2e-6c36263d-255d-4a22-9659-ba12e7965054/file-integrity-daemon) created
    client.go:47: resource type Role with namespace/name (osdk-e2e-6c36263d-255d-4a22-9659-ba12e7965054/file-integrity-operator) created
    client.go:47: resource type Role with namespace/name (osdk-e2e-6c36263d-255d-4a22-9659-ba12e7965054/leader-election-role) created
    client.go:47: resource type ClusterRole with namespace/name (/file-integrity-operator) created
    client.go:47: resource type ClusterRole with namespace/name (/file-integrity-operator-metrics) created
    client.go:47: resource type ClusterRole with namespace/name (/fileintegrity-editor-role) created
    client.go:47: resource type ClusterRole with namespace/name (/fileintegrity-viewer-role) created
    client.go:47: resource type RoleBinding with namespace/name (osdk-e2e-6c36263d-255d-4a22-9659-ba12e7965054/file-integrity-daemon) created
    client.go:47: resource type RoleBinding with namespace/name (osdk-e2e-6c36263d-255d-4a22-9659-ba12e7965054/file-integrity-operator) created
    client.go:47: resource type RoleBinding with namespace/name (osdk-e2e-6c36263d-255d-4a22-9659-ba12e7965054/file-integrity-operator-metrics) created
    client.go:47: resource type RoleBinding with namespace/name (osdk-e2e-6c36263d-255d-4a22-9659-ba12e7965054/leader-election-rolebinding) created
    client.go:47: resource type ClusterRoleBinding with namespace/name (/file-integrity-operator) created
    client.go:47: resource type Deployment with namespace/name (osdk-e2e-6c36263d-255d-4a22-9659-ba12e7965054/file-integrity-operator) created
    helpers.go:282: Initialized cluster resources
    wait_util.go:59: Deployment available (1/1)
    helpers.go:1763: Tainting node: ip-10-0-104-211.ec2.internal
    helpers.go:710: Tainting node failed
--- FAIL: TestFileIntegrityTolerations (30.59s)
=== RUN   TestFileIntegrityLogCompress 

Here is an example patch that causes this issue: https://github.com/openshift/file-integrity-operator/pull/426

Since E2E tests are run serially, this causes the entire suite to cascade in failures.

openshift-bot commented 11 months ago

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close. Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

openshift-bot commented 10 months ago

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity. Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten /remove-lifecycle stale

openshift-bot commented 9 months ago

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen. Mark the issue as fresh by commenting /remove-lifecycle rotten. Exclude this issue from closing again by commenting /lifecycle frozen.

/close

openshift-ci[bot] commented 9 months ago

@openshift-bot: Closing this issue.

In response to [this](https://github.com/openshift/file-integrity-operator/issues/439#issuecomment-1945085214): >Rotten issues close after 30d of inactivity. > >Reopen the issue by commenting `/reopen`. >Mark the issue as fresh by commenting `/remove-lifecycle rotten`. >Exclude this issue from closing again by commenting `/lifecycle frozen`. > >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.