gwen-interpreter / gwen-web

Web automation engine for Gwen
https://gweninterpreter.org
Apache License 2.0
54 stars 18 forks source link

Failure count in report portal is not reflected properly for some cases #94

Closed sse2ebdd closed 2 years ago

sse2ebdd commented 2 years ago

Steps to reproduce -

  1. Create feature file with below content and run it with report portal
    
    Feature: Test

@StepDef Scenario: I define something And something_is_true is defined by javascript "false"

@StepDef Scenario: I check whether something is true And I define something until something_is_true

Scenario: Test And step1 is "step1" And I check whether something is true And step3 is "step3" And step4 is "step4" And step5 is "step5"



2. When using GWEN 3.1.2, in report portal failure count shows blank, which should be 1.
![report-portal-failure-count-issue](https://user-images.githubusercontent.com/86771461/148746577-0c371dd6-5b5a-4cde-ac7d-490f4c04602a.png)

3. I tried to compare junit report generated for GWEN 3.1.2 with GWEN 2.54.1. I found below differences.

 - Extra parameter in <testsuite> tag
GWEN 2.54.1 --> tests="1" errors="1" skipped="0"
GWEN 3.1.2 --> tests="1" errors="1" failures="0" skipped="0"

 - Type of error is also different
GWEN 2.54.1 --> error type="gwen.Errors$StepFailure"
GWEN 3.1.2 --> error type="gwen.core.Errors$StepException"

 - There is "system-err" tag present in GWEN 3.1.2 junit report, which is not present in GWEN 2.54.1.

Env -
GWEN version - 3.1.2
Chrome driver - Latest
bjuric commented 2 years ago

Fixed in gwen-web v3.1.4

sse2ebdd commented 2 years ago

Tested on GWEN 3.1.5. Working as expected.