reportportal / agent-Python-RobotFramework

Robot Framework integration for Report Portal
Apache License 2.0
59 stars 32 forks source link

"Run Keyword And Ignore Error" child keywords status taken into account by Report Portal #194

Open CorentinHache opened 1 month ago

CorentinHache commented 1 month ago

Describe the bug

The "Run Keyword and ignore error" child keywords status are taken into account by report portal. While the error status and messages of these keyword should be ignored by Report Portal, the messages appear in the "Unique Error" and in the Investigation page.

Steps to Reproduce

*** Test Cases ***
Test Report Portal Logs
    Run Keyword And Ignore Error    Failed Keyword
    Run Keyword And Ignore Error    Failed Keyword
    Run Keyword And Ignore Error    Failed Keyword
    Fail    msg=The reason why the test is failed

*** Keywords ***
Failed Keyword
    Fail    msg=To be ignored
Capture d’écran 2024-07-18 à 10 57 23

Expected behavior "Run Keyword and ignore error" child keywords should not be taken into account and not be displayed in the Unique Error and Investigation pages

Actual behavior "Run Keyword and ignore error" child keywords are taken into account and displayed in the Unique Error and Investigation pages

Package versions robotframework_reportportal-5.5.3

CorentinHache commented 1 week ago

The same erroneous behaviour is present for the "Wait Until Keyword Succeeds" keyword :

Describe the bug

In Robot Framework, the "Wait Until Keyword Succeeds" (WUKS) function is used as active delay. This fonction execute several times an action until the action succeeds :

If the action succeeds before a specified duration or a specified number of attempt, the "Wait Until Keyword Succeeds" succeeds. Even if the first attempts of the action have been failed. If the action didn't succeeds after a specified duration or a specified number of attempt, the "Wait Until Keyword Succeeds" fails. Report Portal take into account the error message returned by the failed action inside the WUKS, while it should not.

Steps to Reproduce

Create a Robot Framework script that include a "Wait Until Keyword Succeeds" that is passed after a least one failed attempt. The test should be failed for any reason after the "Wait Until Keyword Succeeds" statement. Push the test result to Report Portal

Expected behavior

Report Portal display the error message that is the cause of the failed test. Actual behavior

Report Portal display the error message that is the cause of the failed test AND the error message of the first attempt of the "Wait Until Keyword Succeeds"

Package versions robotframework_reportportal-5.5.3 ROBOT_FRAMEWORK_VERSION 7.0.0