search

serenity-bdd / serenity-cucumber

Cucumber integration for the Serenity BDD Reporting library
Other
78 stars 74 forks source link

Serenity Test Report is not showing all scenarios #257

Closed punkratz312 closed 3 years ago

punkratz312 commented 3 years ago

I have login feature with 3 scenarios:

image

After clicking on login feature to see the 3 scenarios one is showed up:

image

The other scenarios get executed with errors but they don’t appear on the reports...

Feature: Login

Scenario: User information Given User open relater When User login Then User must enter user information

java.lang.AssertionError: Expecting basic_firstName that is displayed But got: web element is not displayed

at net.serenitybdd.screenplay.ensure.PerformablePredicate.performAs(PerformableExpectations.kt:105)

at net.serenitybdd.screenplay.ensure.PerformablePredicate$ByteBuddy$9l25mJrJ.performAs$accessor$UoCKvutM(Unknown Source)

at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.base/java.lang.reflect.Method.invoke(Method.java:567)

at net.thucydides.core.steps.StepInterceptor.invokeMethod(StepInterceptor.java:492)

at net.thucydides.core.steps.StepInterceptor.executeTestStepMethod(StepInterceptor.java:472)

at net.thucydides.core.steps.StepInterceptor.runTestStep(StepInterceptor.java:445)

at net.thucydides.core.steps.StepInterceptor.runOrSkipMethod(StepInterceptor.java:201)

at net.thucydides.core.steps.StepInterceptor.testStepResult(StepInterceptor.java:188)

at net.thucydides.core.steps.StepInterceptor.intercept(StepInterceptor.java:76)

at net.thucydides.core.steps.ProxyConfiguration$InterceptorDispatcher.intercept(ProxyConfiguration.java:51)

at net.serenitybdd.screenplay.ensure.PerformablePredicate$ByteBuddy$9l25mJrJ.performAs(Unknown Source)

at net.serenitybdd.screenplay.Actor.performTask(Actor.java:252)

at net.serenitybdd.screenplay.Actor.perform(Actor.java:233)

at net.serenitybdd.screenplay.Actor.attemptsTo(Actor.java:178)

at cucumber.starter@1.0.0-SNAPSHOT/starter.stepdefinitions.RelaterStepDefinitions.must_enter_user_information(RelaterStepDefinitions.java:73)

at ?.User must enter user information(file:///Users/vcp/IdeaProjects/relater/02_Software/test_e2e_serenity/src/test/resources/features/login/user_information.feature:6)

| | | | / | | | / | | | /\ | \ | | | | | \ | | | | _ \ | | _ \ | | / \ | / | | | | | |) | || || |/ |_| |/ || // _\ ||\ || |_| |___/

User information(login;user-information)

09:52:38.984 [pool-2-thread-4] INFO n.s.c.w.d.ChromeDriverProvider - Using automatically driver download 09:52:39.642 [pool-2-thread-4] INFO i.g.bonigarcia.wdm.WebDriverManager - Using chromedriver 90.0.4430.24 (resolved driver for Chrome 90) 09:52:39.647 [pool-2-thread-4] INFO i.g.bonigarcia.wdm.WebDriverManager - Exporting webdriver.chrome.driver as /Users/vcp/.cache/selenium/chromedriver/mac64/90.0.4430.24/chromedriver 09:52:40.316 [pool-2-thread-4] INFO n.s.c.w.d.ProvideNewDriver - Instantiating driver 09:52:40.328 [pool-2-thread-4] INFO n.s.c.w.d.ProvideNewDriver - Driver capabilities: Capabilities {acceptInsecureCerts: false, browserName: chrome, chrome.switches: --start-maximized;--test-ty..., goog:chromeOptions: {args: [--start-maximized, --test-type, --no-sandbox, --ignore-certificate-errors;\n, --disable-popup-blocking, --disable-default-apps, --disable-extensions-file-a..., --incognito, --disable-infobars, --disable-gpu, --headless], extensions: []}, loggingPrefs: org.openqa.selenium.logging...} Given User open relater # starter.stepdefinitions.RelaterStepDefinitions.open_relater(net.serenitybdd.screenplay.Actor) When User login # starter.stepdefinitions.RelaterStepDefinitions.login(net.serenitybdd.screenplay.Actor) Then User must enter user information # starter.stepdefinitions.RelaterStepDefinitions.must_enter_user_information(net.serenitybdd.screenplay.Actor) java.lang.AssertionError: Expecting basic_firstName that is displayed But got: web element is not displayed at net.serenitybdd.screenplay.ensure.PerformablePredicate.performAs(PerformableExpectations.kt:105) at net.serenitybdd.screenplay.ensure.PerformablePredicate$ByteBuddy$9l25mJrJ.performAs$accessor$UoCKvutM(Unknown Source) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:567) at net.thucydides.core.steps.StepInterceptor.invokeMethod(StepInterceptor.java:492) at net.thucydides.core.steps.StepInterceptor.executeTestStepMethod(StepInterceptor.java:472) at net.thucydides.core.steps.StepInterceptor.runTestStep(StepInterceptor.java:445) at net.thucydides.core.steps.StepInterceptor.runOrSkipMethod(StepInterceptor.java:201) at net.thucydides.core.steps.StepInterceptor.testStepResult(StepInterceptor.java:188) at net.thucydides.core.steps.StepInterceptor.intercept(StepInterceptor.java:76) at net.thucydides.core.steps.ProxyConfiguration$InterceptorDispatcher.intercept(ProxyConfiguration.java:51) Starting ChromeDriver 90.0.4430.24 (4c6d850f087da467d926e8eddb76550aed655991-refs/branch-heads/4430@{#429}) on port 46665 Only local connections are allowed. Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe. ChromeDriver was started successfully. Apr 16, 2021 9:52:42 AM org.openqa.selenium.remote.ProtocolHandshake createSession INFO: Detected dialect: W3C at net.serenitybdd.screenplay.ensure.PerformablePredicate$ByteBuddy$9l25mJrJ.performAs(Unknown Source) at net.serenitybdd.screenplay.Actor.performTask(Actor.java:252) at net.serenitybdd.screenplay.Actor.perform(Actor.java:233) at net.serenitybdd.screenplay.Actor.attemptsTo(Actor.java:178) at cucumber.starter@1.0.0-SNAPSHOT/starter.stepdefinitions.RelaterStepDefinitions.must_enter_user_information(RelaterStepDefinitions.java:73) at ���.User must enter user information(file:///Users/vcp/IdeaProjects/relater/02_Software/test_e2e_serenity/src/test/resources/features/login/user_information.feature:6)

[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 23.044 s <<< FAILURE! - in Login #3 [ERROR] Login #3.User information Time elapsed: 23.044 s <<< FAILURE! java.lang.AssertionError:

Expecting basic_firstName that is displayed But got: web element is not displayed

[INFO] [INFO] Results: [INFO] [ERROR] Failures: [ERROR]
Expecting basic_firstName that is displayed But got: web element is not displayed [ERROR]
Expecting basic_firstName that is displayed But got: web element is not displayed [INFO] [ERROR] Tests run: 5, Failures: 2, Errors: 0, Skipped: 0 [INFO] [INFO] [INFO] --- serenity-maven-plugin:2.4.4:aggregate (serenity-reports) @ cucumber-starter --- [INFO] Test results for 5 tests generated in 6.2 secs in directory: file:/Users/vcp/IdeaProjects/relater/02_Software/test_e2e_serenity/target/site/serenity/ [INFO] ----------------------------------------- [INFO] SERENITY TESTS : FAILURE [INFO] ----------------------------------------- [INFO] | Tests executed | 5 [INFO] | Tests passed | 3 [INFO] | Tests failed | 2 [INFO] | Tests with errors | 0 [INFO] | Tests compromised | 0 [INFO] | Tests pending | 0 [INFO] | Tests ignored/skipped | 0 [INFO] ------------------------ | -------------- [INFO] | Total Duration | 1m 38s [INFO] | Fastest test took | 16s 878ms [INFO] | Slowest test took | 22s 145ms [INFO] ----------------------------------------- [INFO] [INFO] SERENITY REPORTS [INFO] - Full Report: file:///Users/vcp/IdeaProjects/relater/02_Software/test_e2e_serenity/target/site/serenity/index.html [INFO] [INFO] --- maven-failsafe-plugin:3.0.0-M5:verify (default) @ cucumber-starter --- [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 44.621 s [INFO] Finished at: 2021-04-16T09:53:11+02:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:3.0.0-M5:verify (default) on project cucumber-starter: There are test failures. [ERROR] [ERROR] Please refer to /Users/vcp/IdeaProjects/relater/02_Software/test_e2e_serenity/target/failsafe-reports for the individual test results. [ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Process finished with exit code 1

wakaleo commented 3 years ago

I can't reproduce this, can you provide a sample project?

punkratz312 commented 3 years ago

Same issue:

3 scenarios are shown

image

only 1 scenario is displayed

image.

I provided an sample project facing the same issue:

4 senarios in preview

image

only 1 showing up:

image

maybe the feature names get not merged correctly i don’t know.

punkratz312 commented 3 years ago

here the demo project: https://github.com/punkratz312/serenity_bug

punkratz312 commented 3 years ago

Expecting 5 scnarios:

image

only 1 shows up:

image

punkratz312 commented 3 years ago

its working when putting all scenarios into ne feature.file:

image

so the feature merge in report does not work properly imho or i did something wrong.

image

but of course i want to split big features into multiple feature files and link them by Feature: tag.

wakaleo commented 3 years ago

Feature names need to be unique. A feature file documents a single feature (not a user story or subset of scenarios of a feature). If you want to split or organise the scenarios, group them by Rule or use a directory hierarchy (but don't have multiple feature files with the same name)

punkratz312 commented 3 years ago

my bad, thanks a lot