PVC stuc in deletion test_rbd_capacity_workload_alerts in tearDown

[DanielOsypenko]

https://reportportal-ocs4.apps.ocp-c1.prod.psi.redhat.com/ui/#ocs/launches/557/16353/781887/781989/781990/log?logParams=history%3D781990%26page.page%3D1

Message: failed on setup with "ocs_ci.ocs.exceptions.TimeoutExpiredError: Timed out after 128000.0s running get("", True, None)" Type: None

Text: project = <ocs_ci.ocs.ocp.OCP object at 0x7fbd20b42c40> fio_pvc_dict = {'apiVersion': 'v1', 'kind': 'PersistentVolumeClaim', 'metadata': {'name': 'fio-target'}, 'spec': {'accessModes': ['Re...eOnce'], 'resources': {'requests': {'storage': '1250Gi'}}, 'storageClassName': 'ocs-external-storagecluster-ceph-rbd'}} fio_job_dict = {'apiVersion': 'batch/v1', 'kind': 'Job', 'metadata': {'name': 'fio'}, 'spec': {'backoffLimit': 0, 'template': {'metad...lumes': [{'name': 'fio-target', 'persistentVolumeClaim': {...}}, {'configMap': {...}, 'name': 'fio-config-volume'}]}}}} fio_configmap_dict = {'apiVersion': 'v1', 'data': {'workload.fio': '\n[simple-write]\nreadwrite=write\nbuffered=1\nblocksize=4k\nioengine=libaio\ndirectory=/mnt/target\nnrfiles=8\nfill_fs=1\n'}, 'kind': 'ConfigMap', 'metadata': {'name': 'fio-config'}} measurement_dir = '/tmp/pytest-of-jenkins/pytest-0/measurement_results' tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_rbd_capacity_workload_ale0') supported_configuration = None threading_lock = <unlocked _thread.RLock object owner=0 count=0 at 0x7fbd44be7270>

@pytest.fixture def workload_storageutilization_97p_rbd( project, fio_pvc_dict, fio_job_dict, fio_configmap_dict, measurement_dir, tmp_path, supported_configuration, threading_lock, ): fixture_name = "workload_storageutilization_97p_rbd"

measured_op = workload_fio_storageutilization( fixture_name, project, fio_pvc_dict, fio_job_dict, fio_configmap_dict, measurement_dir, tmp_path, target_percentage=0.97, threading_lock=threading_lock, )

/home/jenkins/workspace/qe-deploy-ocs-cluster-prod/ocs-ci/tests/manage/monitoring/conftest.py:519:

/home/jenkins/workspace/qe-deploy-ocs-cluster-prod/ocs-ci/ocs_ci/ocs/fiojob.py:535: in workload_fio_storageutilization measured_op = measure_operation( /home/jenkins/workspace/qe-deploy-ocs-cluster-prod/ocs-ci/ocs_ci/utility/workloadfixture.py:125: in measure_operation raise (ex) /home/jenkins/workspace/qe-deploy-ocs-cluster-prod/ocs-ci/ocs_ci/utility/workloadfixture.py:113: in measure_operation result = operation() /home/jenkins/workspace/qe-deploy-ocs-cluster-prod/ocs-ci/ocs_ci/ocs/fiojob.py:536: in lambda: write_data_via_fio( /home/jenkins/workspace/qe-deploy-ocs-cluster-prod/ocs-ci/ocs_ci/ocs/fiojob.py:234: in write_data_via_fio pod_name = wait_for_job_completion( /home/jenkins/workspace/qe-deploy-ocs-cluster-prod/ocs-ci/ocs_ci/ocs/fiojob.py:195: in wait_for_job_completion raise (ex) /home/jenkins/workspace/qe-deploy-ocs-cluster-prod/ocs-ci/ocs_ci/ocs/fiojob.py:166: in wait_for_job_completion ocp_pod.wait_for_resource( /home/jenkins/workspace/qe-deploy-ocs-cluster-prod/ocs-ci/ocs_ci/ocs/ocp.py:763: in wait_for_resource raise (ex) /home/jenkins/workspace/qe-deploy-ocs-cluster-prod/ocs-ci/ocs_ci/ocs/ocp.py:654: in wait_for_resource for sample in TimeoutSampler(

self = <ocs_ci.utility.utils.TimeoutSampler object at 0x7fbd3a365970>

def iter(self): if self.start_time is None: self.start_time = time.time() while True: self.last_sample_time = time.time() if self.timeout <= (self.last_sample_time - self.start_time):

      raise self.timeout_exc_cls(*self.timeout_exc_args)

E ocs_ci.ocs.exceptions.TimeoutExpiredError: Timed out after 128000.0s running get("", True, None)

/home/jenkins/workspace/qe-deploy-ocs-cluster-prod/ocs-ci/ocs_ci/utility/utils.py:1288: TimeoutExpiredError Message: failed on teardown with "subprocess.TimeoutExpired: Command '['oc', '--kubeconfig', '/home/jenkins/current-cluster-dir/openshift-cluster-dir/auth/kubeconfig', '-n', 'namespace-test-f5d71d1c305c46829119f746f', 'delete', 'Project', 'namespace-test-f5d71d1c305c46829119f746f']' timed out after 600 seconds" Type: None

=========

We need to check out why PVC is stuck, what storage resources capturing the PVC and support such deletion in any way (try delete 2 times, look for finalizers)

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 30 days if no further activity occurs.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 30 days if no further activity occurs.

[github-actions[bot]]

This issue has been automatically closed due to inactivity. Please re-open if this still requires investigation.

[DanielOsypenko]

still valid

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 30 days if no further activity occurs.