sonic-net / sonic-mgmt

Configuration management examples for SONiC
Other
195 stars 715 forks source link

[test gap] Parametrize 'test_system_health/test_service_checker_with_process_exit' and use conditional_mark to skip containers #7545

Open sanjair-git opened 1 year ago

sanjair-git commented 1 year ago

Description

As part of the test case '_test_service_checker_with_processexit', 'syncd', 'database', 'bgp' and 'swss' containers need to be ignored. 'bgp' and 'swss' containers are ignored due to issue #7151

Currently, the test lists out each container to be ignored as shown in (https://github.com/sonic-net/sonic-mgmt/blob/dac834bfe1385ca165c1788efb3bef7a161d2b20/tests/system_health/test_system_health.py#L123)

The best way to ignore/skip is to parametrize the test case and use conditional_markers to skip the specific containers by tagging the issue specific to those containers. For example, in this case, #7151 issue for 'bgp' and 'swss'. Reference PR #7200

Steps to reproduce the issue: 1. 2. 3.

Describe the results you received:

Describe the results you expected:

Additional information you deem important:

**Output of `show version`:**

```
(paste your output here)
```

**Attach debug file `sudo generate_dump`:**

```
(paste your output here)
```
StormLiangMS commented 4 months ago

Hi @yxieca which team should own this one?

yutongzhang-microsoft commented 3 weeks ago

To achieve this, we need to retrieve all critical processes from DUT, which is determined at run-time, and incorporate the process names into the test case names. However, if we want to apply conditional marks to skip tests, it takes effect during the collection phase, which occurs before the run-time stage.

In other words, since test cases cannot be parameterized before the conditional marks are applied, this creates a logical gap in the testing process. Hence, this test gap doesn't seem feasible.

@yxieca @wangxin Can you help me double confirm if I can close this test gap?