No resources for fio-pod. test_rbd_capacity_workload_alerts FAILS

[DanielOsypenko]

No resources for fio-pod; Test fails: test_rbd_capacity_workload_alerts Happens with Disconnected sometimes and with External mode cluster every time https://reportportal-ocs4.apps.ocp-c1.prod.psi.redhat.com/ui/#ocs/launches/632/19679/955368/955377/log https://reportportal-ocs4.apps.ocp-c1.prod.psi.redhat.com/ui/#ocs/launches/632/19768/960215/960225/log?logParams=history%3D960225%26page.page%3D1

2024-03-16 14:06:40,413 - MainThread - INFO - ocs_ci.ocs.ocp.wait_for_resource.683 - Waiting for a resource(s) of kind Pod identified by name '' using selector None at column name STATUS to reach desired condition Completed 2024-03-16 14:06:40,413 - MainThread - INFO - ocs_ci.utility.utils.exec_cmd.626 - Executing command: oc --kubeconfig /home/jenkins/current-cluster-dir/openshift-cluster-dir/auth/kubeconfig -n namespace-test-0f564895a1424413af83f2970 get Pod -n namespace-test-0f564895a1424413af83f2970 -o yaml 2024-03-16 14:06:40,620 - MainThread - DEBUG - ocs_ci.utility.utils.exec_cmd.667 - Command stdout: apiVersion: v1 items:

• apiVersion: v1 kind: Pod metadata: annotations: openshift.io/scc: restricted-v2 seccomp.security.alpha.kubernetes.io/pod: runtime/default creationTimestamp: "2024-03-16T14:06:40Z" finalizers:
  • batch.kubernetes.io/job-tracking generateName: fio- labels: batch.kubernetes.io/controller-uid: 0f1c9f19-7fb3-4cf1-85c0-fcc005c964bb batch.kubernetes.io/job-name: fio controller-uid: 0f1c9f19-7fb3-4cf1-85c0-fcc005c964bb job-name: fio name: fio-pgxsm namespace: namespace-test-0f564895a1424413af83f2970 ownerReferences:
  • apiVersion: batch/v1 blockOwnerDeletion: true controller: true kind: Job name: fio uid: 0f1c9f19-7fb3-4cf1-85c0-fcc005c964bb resourceVersion: "109105" uid: c98dc91f-203c-42e4-ab90-a88210b56afc spec: containers:
  • command:
    • /usr/bin/fio
    • --output-format=json
    • /etc/fio/workload.fio image: quay.io/fbalak/fio-fedora:latest imagePullPolicy: Always name: fio resources: {} securityContext: allowPrivilegeEscalation: false capabilities: drop:
      • ALL runAsNonRoot: true runAsUser: 1000760000 terminationMessagePath: /dev/termination-log terminationMessagePolicy: File volumeMounts:
    • mountPath: /mnt/target name: fio-target
    • mountPath: /etc/fio name: fio-config-volume
    • mountPath: /var/run/secrets/kubernetes.io/serviceaccount name: kube-api-access-6vnrz readOnly: true dnsPolicy: ClusterFirst enableServiceLinks: true imagePullSecrets:
  • name: default-dockercfg-hmjl7 preemptionPolicy: PreemptLowerPriority priority: 0 restartPolicy: Never schedulerName: default-scheduler securityContext: fsGroup: 1000760000 seLinuxOptions: level: s0:c28,c2 seccompProfile: type: RuntimeDefault serviceAccount: default serviceAccountName: default terminationGracePeriodSeconds: 30 tolerations:
  • effect: NoExecute key: node.kubernetes.io/not-ready operator: Exists tolerationSeconds: 300
  • effect: NoExecute key: node.kubernetes.io/unreachable operator: Exists tolerationSeconds: 300 volumes:
  • name: fio-target persistentVolumeClaim: claimName: fio-target
  • configMap: defaultMode: 420 name: fio-config name: fio-config-volume
  • name: kube-api-access-6vnrz projected: defaultMode: 420 sources:
    • serviceAccountToken: expirationSeconds: 3607 path: token
    • configMap: items:
      • key: ca.crt path: ca.crt name: kube-root-ca.crt
    • downwardAPI: items:
      • fieldRef: apiVersion: v1 fieldPath: metadata.namespace path: namespace
    • configMap: items:
      • key: service-ca.crt path: service-ca.crt name: openshift-service-ca.crt status: conditions:
  • lastProbeTime: null lastTransitionTime: "2024-03-16T14:06:40Z" message: '0/6 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/6 nodes are available: 6 Preemption is not helpful for scheduling..' reason: Unschedulable status: "False" type: PodScheduled phase: Pending qosClass: BestEffort kind: List metadata: resourceVersion: ""

2024-03-16 14:06:40,622 - MainThread - DEBUG - ocs_ci.utility.utils.exec_cmd.676 - Command stderr is empty 2024-03-16 14:06:40,622 - MainThread - DEBUG - ocs_ci.utility.utils.exec_cmd.677 - Command return code: 0 2024-03-16 14:06:40,642 - MainThread - INFO - ocs_ci.utility.utils.exec_cmd.626 - Executing command: oc --kubeconfig /home/jenkins/current-cluster-dir/openshift-cluster-dir/auth/kubeconfig -n namespace-test-0f564895a1424413af83f2970 get Pod fio-pgxsm -n namespace-test-0f564895a1424413af83f2970 2024-03-16 14:06:40,844 - MainThread - DEBUG - ocs_ci.utility.utils.exec_cmd.667 - Command stdout: NAME READY STATUS RESTARTS AGE fio-pgxsm 0/1 Pending 0 0s

---

Failure output in Report Portal:

project = <ocs_ci.ocs.ocp.OCP object at 0x7f48eab448b0> fio_pvc_dict = {'apiVersion': 'v1', 'kind': 'PersistentVolumeClaim', 'metadata': {'name': 'fio-target'}, 'spec': {'accessModes': ['ReadWriteOnce'], 'resources': {'requests': {'storage': '412Gi'}}, 'storageClassName': 'ocs-storagecluster-ceph-rbd'}} fio_job_dict = {'apiVersion': 'batch/v1', 'kind': 'Job', 'metadata': {'name': 'fio'}, 'spec': {'backoffLimit': 0, 'template': {'metad...lumes': [{'name': 'fio-target', 'persistentVolumeClaim': {...}}, {'configMap': {...}, 'name': 'fio-config-volume'}]}}}} fio_configmap_dict = {'apiVersion': 'v1', 'data': {'workload.fio': '\n[simple-write]\nreadwrite=write\nbuffered=1\nblocksize=4k\nioengine=libaio\ndirectory=/mnt/target\nnrfiles=8\nfill_fs=1\n'}, 'kind': 'ConfigMap', 'metadata': {'name': 'fio-config'}} measurement_dir = '/tmp/pytest-of-jenkins/pytest-0/measurement_results' tmp_path = PosixPath('/tmp/pytest-of-jenkins/pytest-0/test_rbd_capacity_workload_ale0') supported_configuration = None threading_lock = <unlocked _thread.RLock object owner=0 count=0 at 0x7f48e86e36c0>

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 30 days if no further activity occurs.

[github-actions[bot]]

This issue has been automatically closed due to inactivity. Please re-open if this still requires investigation.

[DanielOsypenko]

repeats https://reportportal-ocs4.apps.ocp-c1.prod.psi.redhat.com/ui/#ocs/launches/632/20420/981716/981726/log