serenity-bdd / serenity-cucumber

Cucumber integration for the Serenity BDD Reporting library
Other
78 stars 74 forks source link

Serenity Report gives incorrect number of tests run in "Test Result Summary" #106

Closed Ram-Raghu closed 6 years ago

Ram-Raghu commented 6 years ago

Description:

Serenity Report gives incorrect number of tests run in "Test Result Summary"

Steps to Reproduce:

1) Write a Gherkin scenario outline with Examples split into two like in the below example 2) Run the test from maven (mvn clean verify -Dcucumber.options="--tags @add --tags @bank_account) and check the report. This would run the scenario that has @add AND @bank_account tags.

Expected Results:

The "Test Results Summary" should contain the below: Total: 1 Pass: 1

Actual:

The "Test Results Summary" should contain the below. In essence, Serenity takes into account the scenarios from examples which are not run and marking it as pass. Total: 2 Pass: 2

The Related tags section is also inaccurate. The result is inaccurate when large number of tests are run with a scenario outline containing split examples table.

Environment:

1.8.1 1.7.2 1.6.5 **Example**: @add Scenario Outline: Add account Given the account with following details: ,, When I update the acccount to a Then I should be informed of the successful account update @bank_account Examples: Add bank account |accountType|heldBy |servicesSupported | |bankaccount|individual|Deposit,Withdrawal | @card_account Examples: Add card account |accountType|heldBy |servicesSupported | |cardaccount|business |Deposit,Withdrawal | ![image](https://user-images.githubusercontent.com/28475399/33162344-42ee87d8-d07d-11e7-91b7-5acc855bdec6.png)
wakaleo commented 6 years ago

This appears to work correctly for serenity-core 1.9.0-rc.4 and serenity-cucumber 1.6.13

wakaleo commented 6 years ago

Closing for lack of activity