BorisOsipov / wdio-reportportal-reporter

A WebdriverIO plugin. Report results to Report Portal.
MIT License
23 stars 29 forks source link

Skipped scenarios are marked as failed when 'cucumberNestedSteps' is 'true' #173

Closed PriyankaSarangi closed 2 years ago

PriyankaSarangi commented 2 years ago

The problem

I'm using wdio with cucumber and the tests are reporting to a report portal server. When I set the below two combinations of these values 1 - cucumberNestedSteps=true and autoAttachCucumberFeatureToScenario=true 2 - cucumberNestedSteps=true and autoAttachCucumberFeatureToScenario=false Report portal is reporting a scenario as failed in its dashboard when few steps in the scenario are passed and few are skipped. The status of the scenario is skipped in afterScenario hook in wdio.conf.js file How can I mark the scenario as skipped?

Environment

Details

For example if I have a scenario like this: Scenario: scenario headline ✓ Given I open a random website ✓ When I wait on something

Code To Reproduce Issue [ Good To Have ]

Please remember that, with sample code; it's easier to reproduce bug and much faster to fix it.

BorisOsipov commented 2 years ago

Seems reasonable. Feel free to make a PR. Suite status calculates here https://github.com/BorisOsipov/wdio-reportportal-reporter/blob/master/lib/reporter.ts#L158

PriyankaSarangi commented 2 years ago

Also wanted to add that, when a scenario has a status skipped (all of its steps are skipped) then why it is logged into 'To Investigate'. How it can be avoided? I set reporter options as: cucumberNestedSteps=true and autoAttachCucumberFeatureToScenario=true

Screenshot 2022-02-10 at 4 29 54 PM
BorisOsipov commented 2 years ago

@PriyankaSarangi it is default RP behavior. To disable it you have to pass issue with type "NOT_ISSUE" along with endTestItem RQ.

BorisOsipov commented 2 years ago

I mean it requires changes in reporter too

PriyankaSarangi commented 2 years ago

@BorisOsipov EndTestItem is already receiving the issue type as "NOT_ISSUE" including the status of the test (i.e, SKIPPED in this case) from here https://github.com/BorisOsipov/wdio-reportportal-reporter/blob/master/lib/reporter.ts#L236 But still in the report-portal the scenario is tagged as To Investigate

BorisOsipov commented 2 years ago

@PriyankaSarangi it is endTestItem for cucumber step . You see To Investigate for scenario. You have to send issue with type "NOT_ISSUE" for scenario endTestItem RQ too

PriyankaSarangi commented 2 years ago

@BorisOsipov Thank you for your responses. I have created this PR here : https://github.com/BorisOsipov/wdio-reportportal-reporter/pull/175 to address the issue.

BorisOsipov commented 2 years ago

@PriyankaSarangi That's awesome. I will review\merge\release it ETA within a week.

PriyankaSarangi commented 2 years ago

@BorisOsipov Any update on the PR?

BorisOsipov commented 2 years ago

@PriyankaSarangi published as 7.4.1