openbmc / openbmc-test-automation

Apache License 2.0
100 stars 92 forks source link

Skipping test cases using keyword `Skip If` #2175

Closed ruudharing closed 2 years ago

ruudharing commented 2 years ago

With thanks to @generatz for his input.

We noticed that various OpenBMC testcases use a "forced pass" where a "Skip" would be more appropriate. Since Robot Framework 4, there are apparently new keywords: Skip and Skip If . We suggest that they be used whenever testcases are intentionally being skipped.

Examples:

gkeishin commented 2 years ago

we are at an older level in some of the boxes we have robotframework 3.1.2

the Skip If is in higher level robotframework 4.1.3

I ll leave it to the community to move to the latest.. will push up the changes

gkeishin commented 2 years ago

Fix https://gerrit.openbmc-project.xyz/c/openbmc/openbmc-test-automation/+/51906

gkeishin commented 2 years ago
==============================================================================
Test Openbmc Setup :: Test suite to verify if the Robot setup is ready for ...
==============================================================================
Test Redfish Setup :: Verify Redfish works.                           ..#(CST) 2022/03/11 07:42:45.792884 -    0.961918 - Executing: get('/redfish/v1/')
.#(CST) 2022/03/11 07:42:45.817690 -    0.024806 - Executing: delete('/redfish/v1/SessionService/Sessions/rCnd2svXvH')
Test Redfish Setup :: Verify Redfish works.                           | PASS |
------------------------------------------------------------------------------
Test REST Setup :: Verify REST works.                                 | SKIP |
Skipping REST check, user explicitly requested for Redfish.
------------------------------------------------------------------------------
==============================================================================
Test Openbmc Setup :: Test suite to verify if the Robot setup is ready for ...
==============================================================================
Test Redfish Setup :: Verify Redfish works.                           | SKIP |
Skipping Redfish check, user explicitly requested for REST.
------------------------------------------------------------------------------
Test REST Setup :: Verify REST works.                                 .....
 {
    "data": [
        "/xyz/openbmc_project/Chassis",
        "/xyz/openbmc_project/Hiomapd",
        "/xyz/openbmc_project/Ipmi",
        "/xyz/openbmc_project/certs",
        "/xyz/openbmc_project/console",
        "/xyz/openbmc_project/control",
        "/xyz/openbmc_project/dump",
        "/xyz/openbmc_project/events",
        "/xyz/openbmc_project/inventory",
        "/xyz/openbmc_project/ipmi",
        "/xyz/openbmc_project/led",
        "/xyz/openbmc_project/logging",
        "/xyz/openbmc_project/network",
        "/xyz/openbmc_project/object_mapper",
        "/xyz/openbmc_project/sensors",
        "/xyz/openbmc_project/software",
        "/xyz/openbmc_project/state",
        "/xyz/openbmc_project/time",
        "/xyz/openbmc_project/user"
    ],
    "message": "200 OK",
    "status": "ok"
}
Test REST Setup :: Verify REST works.                                 | PASS |
------------------------------------------------------------------------------