Test teardown failed due to leftovers from the test test_nfs_feature_enable_for_ODF_clusters

[jilju]

Test teardown session of the trest case given below failed due to leftovers from the test _test_nfs_feature_enable_for_ODFclusters.

tests/manage/pv_services/test_change_reclaim_policy_of_pv.py::TestChangeReclaimPolicyOfPv::test_change_reclaim_policy_of_pv[CephFileSystem-Retain]

The test _test_nfs_feature_enable_for_ODFclusters enabled nfs feature and and disabled at it's teardown session. After disabling nfs, the test didn't verify whether the resources were removed or not. csi-nfsplugin-provisioner pod was not deleted at the start of the next test which is _test_change_reclaim_policy_ofpv[CephFileSystem-Retain]. Some time later the pod csi-nfsplugin-provisioner was deleted and tests's leftover check observed a resource removal.

Resources removed:
 - '***pods***':
  - apiVersion: v1
    kind: Pod
    metadata:
      annotations:
        k8s.v1.cni.cncf.io/network-status: "[{\n    \"name\": \"openshift-sdn\",\n\
          \    \"interface\": \"eth0\",\n    \"ips\": [\n        \"10.130.2.49\"\n\
          \    ],\n    \"default\": true,\n    \"dns\": {}\n}]"
        k8s.v1.cni.cncf.io/networks-status: "[{\n    \"name\": \"openshift-sdn\",\n\
          \    \"interface\": \"eth0\",\n    \"ips\": [\n        \"10.130.2.49\"\n\
          \    ],\n    \"default\": true,\n    \"dns\": {}\n}]"
        openshift.io/scc: rook-ceph-csi
      creationTimestamp: '2023-01-12T03:53:30Z'
      deletionGracePeriodSeconds: 30
      deletionTimestamp: '2023-01-12T04:03:39Z'
      generateName: csi-nfsplugin-provisioner-5cf4b8c77c-

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 30 days if no further activity occurs.

[github-actions[bot]]

This issue has been automatically closed due to inactivity. Please re-open if this still requires investigation.

[jilju]

Faced the issue in https://reportportal-ocs4.apps.ocp-c1.prod.psi.redhat.com/ui/#ocs/launches/465/21131/1009592/1009736/log

In addition to delay in the deletion of nfs-provisioner pod, some nfs pods still exists after the test test_nfs_feature_enable_for_ODF_clusters.

pod/csi-nfsplugin-holder-ocs-storagecluster-cephcluster-k2xdm pod/csi-nfsplugin-holder-ocs-storagecluster-cephcluster-lstkk pod/csi-nfsplugin-holder-ocs-storagecluster-cephcluster-wwrm4

This test run is from release-4.13 branch.

[jilju]

Adding brown squad label based on the test ownerships.

[ebenahar]

We haven't faced this recently. Closing