Telefonica / toolium

Wrapper tool of Selenium and Appium libraries to test web and mobile applications in a single project
Apache License 2.0
113 stars 61 forks source link

bug(QAWTO-212): fix result for action before the feature error with background #397

Closed ricardogarfe closed 2 months ago

ricardogarfe commented 2 months ago

Description

fix: dynamic environment error output for a feature with:

Current output

Current output scenarios without failure because background is not updated with fail status and this lead to false positives for scenarios:

0 features passed, 1 failed, 0 skipped
0 scenarios passed, 0 failed, 0 skipped, **2 untested**
0 steps passed, 2 failed, 0 skipped, 0 undefined, 2 untested

Corrected output

Scenarios with background fail because actions before the feature fails, console log:

0 features passed, 1 failed, 0 skipped
0 scenarios passed, **2 failed**, 0 skipped
0 steps passed, 2 failed, 0 skipped, 0 undefined

To generate correct report xml output:

<testsuite name="1_fail.Fail feature with a single scenario" tests="2" errors="2"
  failures="0" skipped="0" time="0.0" timestamp="2024-06-28T13:14:40.358338"
  hostname="localhost">
  <testcase classname="1_fail.Fail feature with a single scenario"
    name="First scenario won't be executed due to failed before feature" status="failed" time="0">
    <error type="Exception" message="Preconditions failed">
<![CDATA[
Failing step: Given this step won't be executed ... failed in 0.000s
Location: features/integration/fail.feature:11
FAILED SUB-STEP: Given a failed step
Substep info: Assertion Failed: FAILED SUB-STEP: Given a failed sub-step exception
Substep info: Traceback (most recent call last):
    raise Exception("Sorry, no lucky day!")
]]>
</error>
    <system-out>
<![CDATA[
@scenario.begin

  @regression @sanity
  Scenario: First scenario won't be executed due to failed before feature
    Given this step won't be executed ... failed in 0.000s
    Given I hope this generates test report ... failed in 0.000s

@scenario.end
--------------------------------------------------------------------------------
]]>
</system-out>
  </testcase>
</testsuite>
codeclimate[bot] commented 2 months ago

Code Climate has analyzed commit 52c9ce06 and detected 0 issues on this pull request.

View more on Code Climate.