support rf4 skip status

Change:

Allow for setting SKIP expected status in RF4 the same way you can for FAIL and PASS. See updated README

Testing performed:

ran tests/run.py with 3 test cases added for expected SKIP status results (excluded by tag if RF3)

RF4 and RF3 `tests/run.py` output:

RF4:

[click to expand full test output!]

``` ============================================================================== Tests ============================================================================== Implicit PASS | PASS | ------------------------------------------------------------------------------ Explicit PASS with message :: PASS The message | PASS | The message ------------------------------------------------------------------------------ Explicit SKIP with message :: SKIP The message | SKIP | The message ------------------------------------------------------------------------------ Expected FAIL :: FAIL Expected failure | FAIL | Expected failure ------------------------------------------------------------------------------ SKIP Plus FAIL Expected FAIL :: SKIP FAIL Expected failure | FAIL | Expected failure ------------------------------------------------------------------------------ FAIL Plus SKIP Expected FAIL :: FAIL Expected failure SKIP | FAIL | Expected failure SKIP ------------------------------------------------------------------------------ Ignore documentation before marker :: This text is ignored. FAIL E... | FAIL | Expected failure ------------------------------------------------------------------------------ Expected FAIL with REGEXP :: FAIL REGEXP: Pattern is here.* \d+ | FAIL | Pattern is here multiline 123 ------------------------------------------------------------------------------ Expected FAIL with GLOB :: FAIL GLOB: Globs ??? way *wl even *!?! | FAIL | Globs are way kewl even in multile lines !!! ------------------------------------------------------------------------------ Expected FAIL with STARTS :: FAIL STARTS: This is start | FAIL | This is start and this is end ------------------------------------------------------------------------------ Log message :: LOG 2 Hello world! | PASS | ------------------------------------------------------------------------------ Log message with message index :: LOG 2:1 Hello LOG 2:2 world LOG ... | PASS | ------------------------------------------------------------------------------ Log messages with levels :: LOG 2 DEBUG Hello LOG 3 WARN World LOG... [ WARN ] World [ ERROR ] Tidii | PASS | ------------------------------------------------------------------------------ Trailing and leading whitespace is ignored in log messages :: LOG ... | PASS | ------------------------------------------------------------------------------ Log messages deeper :: LOG 2:1 Hello LOG 2:2 World LOG 3.1 DEBUG U... | PASS | ------------------------------------------------------------------------------ Log messages deeper with setup :: LOG 1:1 Hello LOG 1:2 World LOG ... | PASS | ------------------------------------------------------------------------------ Log messages deeper with wildcard :: LOG 2:1 Hello LOG 2:2 ANY Wor... | PASS | ------------------------------------------------------------------------------ Log messages deeper with wildcard and setup :: LOG 1:1 Hello LOG 1... | PASS | ------------------------------------------------------------------------------ Log message with REGEXP :: LOG 2 REGEXP: H[ei]l{2}o w\w+! LOG 2 RE... | PASS | ------------------------------------------------------------------------------ Log message with GLOB :: LOG 2 GLOB: *world! LOG 2 GLOB: Hell? ***! | PASS | ------------------------------------------------------------------------------ Log message with STARTS :: LOG 2 STARTS: Hello LOG 2 STARTS: Hell ... | PASS | ------------------------------------------------------------------------------ NONE log message :: LOG 2 NONE LOG 2:1 NONE LOG 3:1 Message LOG 3:... | PASS | ------------------------------------------------------------------------------ Test Setup Check Is Done By SETUP Marker :: ... LOG SETUP:1 NONE L... | PASS | ------------------------------------------------------------------------------ Error When No Setup :: ... LOG SETUP.1:1 PASS LOG 2:1 KALA | PASS | ------------------------------------------------------------------------------ Test Setup Check Is Done By SETUP Marker and wildcard is used :: .... | PASS | ------------------------------------------------------------------------------ Error When No Setup and wildcard is used :: ... LOG SETUP.1:* PASS... | PASS | ------------------------------------------------------------------------------ Test Teardown Check Is Done By TEARDOWN Marker :: ... LOG TEARDOWN... | PASS | ------------------------------------------------------------------------------ Error When No Teardown :: LOG TEARDOWN:1 foobar | PASS | ------------------------------------------------------------------------------ Test Teardown Check Is Done By TEARDOWN Marker and wildcard is use... | PASS | ------------------------------------------------------------------------------ Error When No Teardown and wildcard is used :: LOG TEARDOWN:* foobar | PASS | ------------------------------------------------------------------------------ Error When NONE is used with wildcard :: LOG 2.1:* INFO NONE | PASS | ------------------------------------------------------------------------------ Expected FAIL and log messages :: This text is ignored. FAIL Told ... | FAIL | Told ya!! ------------------------------------------------------------------------------ Expected PASS and log messages :: This text is ignored. PASS Told ... | PASS | Told ya!! ------------------------------------------------------------------------------ Expected PASS and teadown does not affect :: This text is ignored.... | PASS | ------------------------------------------------------------------------------ FAILURE: Unexpected PASS :: FAIL Expected failure does not occur | PASS | ------------------------------------------------------------------------------ FAILURE: Wrong PASS message | PASS | Unexpected message ------------------------------------------------------------------------------ FAILURE: Unexpected FAIL | FAIL | Unexpected error message ------------------------------------------------------------------------------ FAILURE: Wrong message :: FAIL Expected failure | FAIL | Not the expected message ------------------------------------------------------------------------------ FAILURE: Wrong log message :: LOG 2 Hello world! | PASS | ------------------------------------------------------------------------------ FAILURE: Wrong log level :: LOG 2.1 Hello world! | PASS | ------------------------------------------------------------------------------ FAILURE: Unexpected log message :: LOG 2.1:2 NONE | PASS | ------------------------------------------------------------------------------ FAILURE: Non-existing keyword :: LOG 2 No keyword here | PASS | ------------------------------------------------------------------------------ FAILURE: Non-existing log message :: LOG 2:2 No message here | PASS | ------------------------------------------------------------------------------ FAILURE: Non-existing log message wildcard :: LOG 1:* Bogus message | PASS | ------------------------------------------------------------------------------ Tests | FAIL | 44 tests, 33 passed, 10 failed, 1 skipped ============================================================================== Output: /Users/ejones/Documents/workspace/statuschecker/test/results/output.xml Checking /Users/ejones/Documents/workspace/statuschecker/test/results/output.xml Log: /Users/ejones/Documents/workspace/statuschecker/test/results/log.html Report: /Users/ejones/Documents/workspace/statuschecker/test/results/report.html All 44 tests passed/failed/logged/skipped as expected. Run on CPython 3.9.7. Robot Framework version: 4.0 ```

yes|pip uninstall robotframework && pip install robotframework==4; python test/run.py
[...]
All 44 tests passed/failed/logged/skipped as expected.
Run on CPython 3.9.7.
Robot Framework version: 4.0

RF3:

[click to expand full test output!]

``` ============================================================================== Tests ============================================================================== Implicit PASS | PASS | ------------------------------------------------------------------------------ Explicit PASS with message :: PASS The message | PASS | The message ------------------------------------------------------------------------------ Expected FAIL :: FAIL Expected failure | FAIL | Expected failure ------------------------------------------------------------------------------ Ignore documentation before marker :: This text is ignored. FAIL E... | FAIL | Expected failure ------------------------------------------------------------------------------ Expected FAIL with REGEXP :: FAIL REGEXP: Pattern is here.* \d+ | FAIL | Pattern is here multiline 123 ------------------------------------------------------------------------------ Expected FAIL with GLOB :: FAIL GLOB: Globs ??? way *wl | FAIL | Globs are way kewl even in multile lines !!! ------------------------------------------------------------------------------ Expected FAIL with STARTS :: FAIL STARTS: This is start | FAIL | This is start and this is end ------------------------------------------------------------------------------ Log message :: LOG 2 Hello world! | PASS | ------------------------------------------------------------------------------ Log message with message index :: LOG 2:1 Hello LOG 2:2 world LOG ... | PASS | ------------------------------------------------------------------------------ Log messages with levels :: LOG 2 DEBUG Hello [ WARN ] World [ ERROR ] Tidii | PASS | ------------------------------------------------------------------------------ Trailing and leading whitespace is ignored in log messages :: LOG ... | PASS | ------------------------------------------------------------------------------ Log messages deeper :: LOG 2:1 Hello | PASS | ------------------------------------------------------------------------------ Log messages deeper with setup :: LOG 1:1 Hello | PASS | ------------------------------------------------------------------------------ Log messages deeper with wildcard :: LOG 2:1 Hello | PASS | ------------------------------------------------------------------------------ Log messages deeper with wildcard and setup :: LOG 1:1 Hello | PASS | ------------------------------------------------------------------------------ Log message with REGEXP :: LOG 2 REGEXP: H[ei]l{2}o w\w+! LOG 2 RE... | PASS | ------------------------------------------------------------------------------ Log message with GLOB :: LOG 2 GLOB: *world! LOG 2 GLOB: Hell? ***! | PASS | ------------------------------------------------------------------------------ Log message with STARTS :: LOG 2 STARTS: Hello LOG 2 STARTS: Hell | PASS | ------------------------------------------------------------------------------ NONE log message :: LOG 2 NONE LOG 2:1 NONE LOG 3:1 Message LOG 3:... | PASS | ------------------------------------------------------------------------------ Test Setup Check Is Done By SETUP Marker :: LOG SETUP:1 NONE | PASS | ------------------------------------------------------------------------------ Error When No Setup :: LOG SETUP.1:1 PASS | PASS | ------------------------------------------------------------------------------ Test Setup Check Is Done By SETUP Marker and wildcard is used :: L... | PASS | ------------------------------------------------------------------------------ Error When No Setup and wildcard is used :: LOG SETUP.1:* PASS | PASS | ------------------------------------------------------------------------------ Test Teardown Check Is Done By TEARDOWN Marker :: LOG TEARDOWN:1 f... | PASS | ------------------------------------------------------------------------------ Error When No Teardown :: LOG TEARDOWN:1 foobar | PASS | ------------------------------------------------------------------------------ Test Teardown Check Is Done By TEARDOWN Marker and wildcard is use... | PASS | ------------------------------------------------------------------------------ Error When No Teardown and wildcard is used :: LOG TEARDOWN:* foobar | PASS | ------------------------------------------------------------------------------ Error When NONE is used with wildcard :: LOG 2.1:* INFO NONE | PASS | ------------------------------------------------------------------------------ Expected FAIL and log messages :: This text is ignored. FAIL Told ... | FAIL | Told ya!! ------------------------------------------------------------------------------ Expected PASS and log messages :: This text is ignored. PASS Told ... | PASS | Told ya!! ------------------------------------------------------------------------------ Expected PASS and teadown does not affect :: This text is ignored. | PASS | ------------------------------------------------------------------------------ FAILURE: Unexpected PASS :: FAIL Expected failure does not occur | PASS | ------------------------------------------------------------------------------ FAILURE: Wrong PASS message | PASS | Unexpected message ------------------------------------------------------------------------------ FAILURE: Unexpected FAIL | FAIL | Unexpected error message ------------------------------------------------------------------------------ FAILURE: Wrong message :: FAIL Expected failure | FAIL | Not the expected message ------------------------------------------------------------------------------ FAILURE: Wrong log message :: LOG 2 Hello world! | PASS | ------------------------------------------------------------------------------ FAILURE: Wrong log level :: LOG 2.1 Hello world! | PASS | ------------------------------------------------------------------------------ FAILURE: Unexpected log message :: LOG 2.1:2 NONE | PASS | ------------------------------------------------------------------------------ FAILURE: Non-existing keyword :: LOG 2 No keyword here | PASS | ------------------------------------------------------------------------------ FAILURE: Non-existing log message :: LOG 2:2 No message here | PASS | ------------------------------------------------------------------------------ FAILURE: Non-existing log message wildcard :: LOG 1:* Bogus message | PASS | ------------------------------------------------------------------------------ Tests | FAIL | 41 critical tests, 33 passed, 8 failed 41 tests total, 33 passed, 8 failed ============================================================================== Output: /Users/ejones/Documents/workspace/statuschecker/test/results/output.xml Checking /Users/ejones/Documents/workspace/statuschecker/test/results/output.xml Log: /Users/ejones/Documents/workspace/statuschecker/test/results/log.html Report: /Users/ejones/Documents/workspace/statuschecker/test/results/report.html All 41 tests passed/failed/logged as expected. Run on CPython 3.9.7. Robot Framework version: 3.0 ```

yes|pip uninstall robotframework && pip install robotframework==3; python test/run.py
[...]
All 41 tests passed/failed/logged as expected.
Run on CPython 3.9.7.
Robot Framework version: 3.0

robotframework / statuschecker