search

canonical / hotsos

Software analysis toolkit. Define checks in high-level language and leverage library to perform analysis of common Cloud applications.
Apache License 2.0
30 stars 37 forks source link

Enhancement/detect missing scenario tests #839

Closed mustafakemalgilor closed 2 months ago

mustafakemalgilor commented 2 months ago

Each scenario should have at least one test file, but we had no way of checking that. This patch introduces a new test case named test_scenarios_check_mapping that discovers all scenario tests and the scenarios, then finds out which scenarios doesn't have a test case. The test case also verifies that every scenario has the checks and conclusions sections as well.

Added test_yscenario_ to the test cases generated from YAML files to be able to distinguish them in a test run._

mustafakemalgilor commented 2 months ago

The CI will fail, because there are scenarios without test cases, which the PR is all about finding out them and failing if so. I'll add test cases for each of them now.

mustafakemalgilor commented 2 months ago

The CI will fail, because there are scenarios without test cases, which the PR is all about finding out them and failing if so. I'll add test cases for each of them now.

ATM, the test case reports that following scenarios do not have a test case:

    "/workspace/github/hotsos/hotsos/defs/scenarios/openvswitch/ovn/ovn_central_services.yaml",
    "/workspace/github/hotsos/hotsos/defs/scenarios/storage/ceph/ceph-osd/osd_slow_ops.yaml",
    "/workspace/github/hotsos/hotsos/defs/scenarios/storage/ceph/ceph-osd/osd_flapping.yaml",
    "/workspace/github/hotsos/hotsos/defs/scenarios/openstack/neutron/bugs/lp1883089.yaml",
    "/workspace/github/hotsos/hotsos/defs/scenarios/openstack/nova/bugs/lp1944619.yaml",
    "/workspace/github/hotsos/hotsos/defs/scenarios/openstack/nova/bugs/lp1888395.yaml",
    "/workspace/github/hotsos/hotsos/defs/scenarios/openstack/nova/bugs/lp1860743.yaml"
mustafakemalgilor commented 2 months ago

The CI will fail, because there are scenarios without test cases, which the PR is all about finding out them and failing if so. I'll add test cases for each of them now.

ATM, the test case reports that following scenarios do not have a test case:

    "/workspace/github/hotsos/hotsos/defs/scenarios/openvswitch/ovn/ovn_central_services.yaml",
    "/workspace/github/hotsos/hotsos/defs/scenarios/storage/ceph/ceph-osd/osd_slow_ops.yaml",
    "/workspace/github/hotsos/hotsos/defs/scenarios/storage/ceph/ceph-osd/osd_flapping.yaml",
    "/workspace/github/hotsos/hotsos/defs/scenarios/openstack/neutron/bugs/lp1883089.yaml",
    "/workspace/github/hotsos/hotsos/defs/scenarios/openstack/nova/bugs/lp1944619.yaml",
    "/workspace/github/hotsos/hotsos/defs/scenarios/openstack/nova/bugs/lp1888395.yaml",
    "/workspace/github/hotsos/hotsos/defs/scenarios/openstack/nova/bugs/lp1860743.yaml"

Opened PRs for each of these. This MP's CI should be green after the following MP's have been merged:

842 #843 #844 #845 #846 #848 #849

mustafakemalgilor commented 2 months ago

The CI will fail, because there are scenarios without test cases, which the PR is all about finding out them and failing if so. I'll add test cases for each of them now.

ATM, the test case reports that following scenarios do not have a test case:

    "/workspace/github/hotsos/hotsos/defs/scenarios/openvswitch/ovn/ovn_central_services.yaml",
    "/workspace/github/hotsos/hotsos/defs/scenarios/storage/ceph/ceph-osd/osd_slow_ops.yaml",
    "/workspace/github/hotsos/hotsos/defs/scenarios/storage/ceph/ceph-osd/osd_flapping.yaml",
    "/workspace/github/hotsos/hotsos/defs/scenarios/openstack/neutron/bugs/lp1883089.yaml",
    "/workspace/github/hotsos/hotsos/defs/scenarios/openstack/nova/bugs/lp1944619.yaml",
    "/workspace/github/hotsos/hotsos/defs/scenarios/openstack/nova/bugs/lp1888395.yaml",
    "/workspace/github/hotsos/hotsos/defs/scenarios/openstack/nova/bugs/lp1860743.yaml"

Opened PRs for each of these. This MP's CI should be green after the following MP's have been merged:

842 #843 #844 #845 #846 #848 #849

@dosaboy

All said MP's are merged. Rebased this MP to main, and the pipeline is now green. Ready for review.