test_ceph_rgw_metrics_after_metrics_exporter_respin is failing on IBM Power

[Pooja-Soni78]

tests/manage/monitoring/prometheusmetrics/test_rgw.py::test_ceph_rgw_metrics_after_metrics_exporter_respin test case is failing due to following error -

E       AssertionError: OCS Monitoring should provide some value(s) for tested rgw metrics, so that the list of metrics without results is empty.
E       assert ['ceph_rgw_ql...t_count', ...] == []
E         Left contains 16 more items, first extra item: 'ceph_rgw_qlen'
E         Full diff:
E           [
E         -  ,
E         +  'ceph_rgw_qlen',
E         +  'ceph_rgw_req',
E         +  'ceph_rgw_cache_miss',...
E         
E         ...Full output truncated (15 lines hidden), use '-vv' to show

[Pooja-Soni78]

_____________ test_ceph_rgw_metrics_after_metrics_exporter_respin ______________

rgw_deployments = [{'apiVersion': 'apps/v1', 'kind': 'Deployment', 'metadata': {'annotations': {'banzaicloud.com/last-applied': 'UEsDBBQ...-cephobjectstore', 'rgw': 'ocs-storagecluster-cephobjectstore', 'rook_cluster': 'openshift-storage', ...}}, ...}, ...}]

    @skipif_ocs_version("<4.6")
    @tier4c
    @pytest.mark.polarion_id("OCS-2385")
    @skipif_managed_service
    def test_ceph_rgw_metrics_after_metrics_exporter_respin(rgw_deployments):
        """
        RGW metrics should be provided via OCP Prometheus even after
        ocs-metrics-exporter pod is respinned.

        """
        logger.info("Respin ocs-metrics-exporter pod")
        pod_obj = ocp.OCP(
            kind=constants.POD, namespace=config.ENV_DATA["cluster_namespace"]
        )
        metrics_pods = pod_obj.get(selector="app.kubernetes.io/name=ocs-metrics-exporter")[
            "items"
        ]
        assert len(metrics_pods) == 1
        metrics_pod_data = metrics_pods[0]
        metrics_pod = OCS(**metrics_pod_data)
        metrics_pod.delete(force=True)

        logger.info("Wait for ocs-metrics-exporter pod to come up")
        assert pod_obj.wait_for_resource(
            condition="Running",
            selector="app.kubernetes.io/name=ocs-metrics-exporter",
            resource_count=1,
            timeout=600,
        )

        logger.info("Collect RGW metrics")
        prometheus = PrometheusAPI()
        list_of_metrics_without_results = metrics.get_missing_metrics(
            prometheus, metrics.ceph_rgw_metrics
        )
        msg = (
            "OCS Monitoring should provide some value(s) for tested rgw metrics, "
            "so that the list of metrics without results is empty."
        )
>       assert list_of_metrics_without_results == [], msg
E       AssertionError: OCS Monitoring should provide some value(s) for tested rgw metrics, so that the list of metrics without results is empty.
E       assert ['ceph_rgw_ql...t_count', ...] == []
E         Left contains 16 more items, first extra item: 'ceph_rgw_qlen'
E         Full diff:
E           [
E         -  ,
E         +  'ceph_rgw_qlen',
E         +  'ceph_rgw_req',
E         +  'ceph_rgw_cache_miss',...
E         
E         ...Full output truncated (15 lines hidden), use '-vv' to show

tests/manage/monitoring/prometheusmetrics/test_rgw.py:61: AssertionError

logs - test_ceph_rgw_metrics_after_metrics_exporter_respin.log

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 30 days if no further activity occurs.

[github-actions[bot]]

This issue has been automatically closed due to inactivity. Please re-open if this still requires investigation.