search

serenity-bdd / serenity-cucumber

Cucumber integration for the Serenity BDD Reporting library
Other
78 stars 74 forks source link

The same reports were generated multiple times when using Scenario Outline #170

Closed starkhorn closed 5 years ago

starkhorn commented 5 years ago

Hello,

Not sure if this is intended or not but while I was trying to improve the test execution time, I found that Serenity called a method to generate the same report more than 1 time by looking at the console logs produced:

15:21:23.291 [pool-2-thread-1] DEBUG n.t.c.r.h.HtmlAcceptanceTestReporter - GENERATE TEST OUTCOME REPORT FOR Order a cappuccino in f8f40086c46f593ecfef4b4d9af5ff156e875dd1d6c8ac44ce3d4f960a1a4a2e
15:21:23.297 [pool-2-thread-2] DEBUG n.t.c.r.h.HtmlAcceptanceTestReporter - GENERATE TEST OUTCOME REPORT FOR Order a latte in 758d93aec3819eb26dbea66a409da75b49fc8b11c2d1c7cd055385bdcdd3ef11
15:21:24.413 [pool-4-thread-1] DEBUG n.t.c.r.h.HtmlAcceptanceTestReporter - GENERATE TEST OUTCOME REPORT FOR Order a cappuccino in f8f40086c46f593ecfef4b4d9af5ff156e875dd1d6c8ac44ce3d4f960a1a4a2e
15:21:24.421 [pool-4-thread-2] DEBUG n.t.c.r.h.HtmlAcceptanceTestReporter - GENERATE TEST OUTCOME REPORT FOR Order a latte in 758d93aec3819eb26dbea66a409da75b49fc8b11c2d1c7cd055385bdcdd3ef11
15:21:24.531 [pool-4-thread-3] DEBUG n.t.c.r.h.HtmlAcceptanceTestReporter - GENERATE TEST OUTCOME REPORT FOR Order an americano in 080a5b028919a6997b162f78f34ef824f2031c5d392270b8497850c1017f4177

The feature file used to demonstrate the issue is:

Feature: Order a coffee

  Scenario: Order a cappuccino
    Given I am thirsty
    When I order a cappuccino
    Then I should not receive a latte

  Scenario Outline: Order a latte
    Given I am thirsty
    When I order a latte
    Then I should not receive a "<coffee>"

    Examples:
      | coffee     |
      | cappuccino |

  Scenario: Order an americano
    Given I am thirsty
    When I order a americano
    Then I should not receive a latte

It seems the report generation was called after a scenario outline finished running all its examples as described in SerenityReporter.finishExample()

Here is the information about the library version used:

Library Version
serenity-core 1.9.26
serenity-cucumber 1.9.8

Here is also the repo I used to create the issue: https://github.com/starkhorn/serenity-cucumber-starter

wakaleo commented 5 years ago

Possibly a bug, thanks for pointing it out.