search

vividus-framework / vividus

VIVIDUS is all in one test automation tool
https://docs.vividus.dev
Apache License 2.0
359 stars 85 forks source link

Implement JBehave-JUnit-Runner for Vividus #438

Open valfirst opened 4 years ago

valfirst commented 4 years ago

Current jbehave-junit-runner is strongly relies on jbehave-core and can't take into account Vividus features like batches, per-batch configuration of meta-filters and etc.

Possible solutions:

dKatechev commented 4 years ago

Scenarios with Examples table are not running properly with Eclipse JUnit 4 test runner. However, tests are running successfully with gradle task runStories

Scenario example:

======

Description: description

Meta: @group group @feature feature

Scenario: scenario 1 Meta: @testCaseId testCaseId @story story @autoStory autoStory Given request body: 'body' When I set request headers: |name |value | |Content-Type|application/json| |accept |application/json| When I issue a HTTP POST request for a resource with the URL '${ig.mas.api.host}${ig.mas.api.management}' Then a JSON element by the JSON path '$.atgOrderId' is equal to '"20064410069"' Then a JSON element by the JSON path '$.status' is equal to 'ATG_ACKNOWLEDGED'

Scenario: scenario 2 Meta: @testCaseId tcId When I issue a HTTP POST request for a resource with the URL 'someUrl' Then a JSON element by the JSON path '$.errors[0].errorCode' is equal to '' Examples: |paramName |paramValue |errorCode |errorMessage | |email_with_value|"email": ""|"ERR_M_400000"|"Invalid format"|

======

Test execution is failing with the exception below:

2020-04-21 13:54:22,476 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingStoryReporter - Description: description
(file:/D:/crepo/commerce-api/v-seph/instagram-tests/bin/main/story/seph/instagram/management/CE-301 Management Service - Accept Order Record.story)
2020-04-21 13:54:22,477 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingStoryReporter - Meta:
2020-04-21 13:54:22,478 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingStoryReporter - @feature feature
2020-04-21 13:54:22,478 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingStoryReporter - @group group
2020-04-21 13:54:22,499 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - Scenario 'scenario 1' excluded by filter: groovy: (testCaseId == 'tcId' && !skip)
2020-04-21 13:54:22,500 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingStoryReporter - Scenario: scenario 2
2020-04-21 13:54:22,501 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingStoryReporter - Meta:
2020-04-21 13:54:22,501 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingStoryReporter - @testCaseId tcId
2020-04-21 13:54:22,501 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingStoryReporter - Examples:
2020-04-21 13:54:22,502 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingStoryReporter - When I issue a HTTP POST request for a resource with the URL 'someUrl'
2020-04-21 13:54:22,502 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingStoryReporter - Then a JSON element by the JSON path '$.errors[0].errorCode' is equal to '<errorCode>'
2020-04-21 13:54:22,504 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingStoryReporter - |paramName|paramValue|errorCode|errorMessage|
|email_with_value|"email": ""|"ERR_M_400000"|"Invalid format"|
2020-04-21 13:54:22,504 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - Failed to run story file:/D:/crepo/commerce-api/v-seph/instagram-tests/bin/main/story/seph/instagram/management/CE-301 Management Service - Accept Order Record.story
2020-04-21 13:54:22,505 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - java.lang.NullPointerException
2020-04-21 13:54:22,506 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at com.github.valfirst.jbehave.junit.monitoring.JUnitScenarioReporter$TestState.getNextOrNull(JUnitScenarioReporter.java:408)
2020-04-21 13:54:22,506 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at com.github.valfirst.jbehave.junit.monitoring.JUnitScenarioReporter$TestState.moveToNextExample(JUnitScenarioReporter.java:392)
2020-04-21 13:54:22,506 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at com.github.valfirst.jbehave.junit.monitoring.JUnitScenarioReporter$TestState.access$1300(JUnitScenarioReporter.java:368)
2020-04-21 13:54:22,506 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at com.github.valfirst.jbehave.junit.monitoring.JUnitScenarioReporter.example(JUnitScenarioReporter.java:234)
2020-04-21 13:54:22,506 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at org.jbehave.core.reporters.DelegatingStoryReporter.example(DelegatingStoryReporter.java:143)
2020-04-21 13:54:22,507 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at org.jbehave.core.reporters.ConcurrentStoryReporter.example(ConcurrentStoryReporter.java:324)
2020-04-21 13:54:22,507 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at org.jbehave.core.reporters.DelegatingStoryReporter.example(DelegatingStoryReporter.java:143)
2020-04-21 13:54:22,507 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at org.jbehave.core.embedder.PerformableTree$ExamplePerformableScenario.perform(PerformableTree.java:1246)
2020-04-21 13:54:22,507 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at org.jbehave.core.embedder.PerformableTree$PerformableScenario.perform(PerformableTree.java:1095)
2020-04-21 13:54:22,507 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at org.jbehave.core.embedder.PerformableTree$PerformableStory.performScenarios(PerformableTree.java:987)
2020-04-21 13:54:22,507 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at org.jbehave.core.embedder.PerformableTree$PerformableStory.perform(PerformableTree.java:955)
2020-04-21 13:54:22,507 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at org.jbehave.core.embedder.PerformableTree.performCancellable(PerformableTree.java:471)
2020-04-21 13:54:22,507 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at org.jbehave.core.embedder.PerformableTree.perform(PerformableTree.java:438)
2020-04-21 13:54:22,507 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at org.jbehave.core.embedder.StoryManager$EnqueuedStory.call(StoryManager.java:293)
2020-04-21 13:54:22,507 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at org.jbehave.core.embedder.StoryManager$EnqueuedStory.call(StoryManager.java:266)
2020-04-21 13:54:22,507 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
2020-04-21 13:54:22,508 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
2020-04-21 13:54:22,508 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
2020-04-21 13:54:22,508 [batch-1-thread-1] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - at java.base/java.lang.Thread.run(Thread.java:834)
2020-04-21 13:54:22,673 [main] INFO  org.vividus.bdd.log.LoggingEmbedderMonitor - 8 stories excluded by filter: groovy: (testCaseId == 'tcId' && !skip)