Closed gouthamvarma1 closed 6 years ago
Where are you defining the tags?
I am also facing the same issue. The tags are defined above the Scenario Outline.
Eg:
@ListCustomer @Regression @TestCase6
Scenario Outline: Verify LIST Customer - By StartIndex and Resultsperpage is NEGATIVE
Given the List customer API
And Verify the list customer is displaying all the customers By StartIndex<startIndex> and Resultsperpage<ResultsPerPage>
Then the response code is <Response Code>
Examples:
|startIndex |ResultsPerPage | Response Code |
|4 |-588 | 400 |
|2 |-854 | 400 |
|5 |-66 | 400 |
|3 |-2 | 400 |
How are you defining the tags when you run the tests?
@wakaleo the tags are defined in serenity runner class currently and executed via maven with command clean verify
Example:
package test;
import cucumber.api.CucumberOptions;
import net.serenitybdd.cucumber.CucumberWithSerenity;
import org.junit.runner.RunWith;
@RunWith(CucumberWithSerenity.class)
@CucumberOptions(plugin = { "pretty" }, features = "src/test/resources/features", tags = { "@regression3,@api3" })
public class CucumberTestSuite {
}
The annotation should be:
@RunWith(CucumberWithSerenity.class)
@CucumberOptions(
plugin = { "pretty" },
features = "src/test/resources/features",
tags = { "@regression3","@api3" }
)
public class CucumberTestSuite {
}
@wakaleo : Please correct me If I am wrong
I have a feature file with the below scenarios @TC1 Scenario: Test1 @TC2 Scenari2: Test2
If I want to run both the scenarios my tags will be like tags = {"@TC1,@TC2"} //then both scenarios are executed but the report is not recorded.( Report Generated with Empty/no data )
But If I use tags
tags = {"@TC1","@TC2"} //none of the scenario's executed
Yes, As far as I can tell, tags = {"@TC1","@TC2"}
will only run scenarios with both tags, and tags = {"@TC1,@TC2"}
(which I have never used) runs scenarios with either tag. You would need to check the Cucumber documentation to be sure. For any issues, the @CucumberOptions annotation is managed by the Cucumber-JVM team, not by the Serenity team, so it would be better to raise an issue with them directly.
Hi @wakaleo , I tried the same scenario as mentioned above. I see that the tests are getting executed and report is generated with no data in it. From this I assume that the issue is with the reporting feature of Serenity by using OR condition.
I'm not seeing any issues in execution or reporting with the latest versions of serenity-core and serenity-cucumber, using with @CucumberOptions or the cucumber.options property passed in on the command line. Can you produce a sample project that reproduces the issue?
@wakaleo I have attached a sample project to reproduce the issue Upon execution in eclipse with build parameters clean verify I see the below output
[INFO] Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.235 s - in Feature: Example
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 20, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- serenity-maven-plugin:1.9.0-rc.3:aggregate (serenity-reports) @ IssueOR ---
[INFO] current_project.base.dir: C:\Users\gpendyala\eclipse-workspace-FedEx\IssueOR
[INFO] Generating test results for 0 tests
[INFO] 0 requirements loaded after 126 ms
[INFO] 0 related requirements found after 126 ms
[INFO] Generating test outcome reports: false
[INFO] Starting generating reports: 139 ms
[INFO] Configured report threads: 40
[INFO] Test results for 0 tests generated in 582 ms
[INFO]
[INFO] --- maven-failsafe-plugin:2.20:verify (default) @ IssueOR ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8.891 s
[INFO] Finished at: 2018-03-02T17:41:38+05:30
[INFO] Final Memory: 46M/566M
[INFO] ------------------------------------------------------------------------
hi @gouthamvarma1. Currently serenity supports only the "new style" cucumber tags. Please use tags={"@smoketest or @regressiontest"} instead of tags={"@smoketest,@regressiontest"} . For a complete guide of migrating from the old style tags, please check this page : https://github.com/cucumber/cucumber/tree/master/tag-expressions
Closing this as we will only be supporting the new style Cucumber tags.
I had the same problem, solved it using "cucumber.options" instead of "tags". Examples:
Ok - Generating test results for n tests mvn -Dcucumber.options="--tags ''" clean install mvn -Dcucumber.options="--tags '@Tag1'" clean install mvn -Dcucumber.options="--tags @Tag1" clean install
Error Generating test results for 0 tests mvn -Dtags=@Tag1 clean install
ORing: Tags which are comma separated are ORed.
Results are not recorded for tags when using ORing Tags Used tags={"@SmokeTest,@RegressionTest"} --- Reports are not recorded for this tags expression
It will not always simple like executing a single tag. It can be complicated like executing scenarios which are tagged either as @SmokeTest or @RegressionTest
Feature file Example :
@SmokeTest @test @all Scenario: Test scenario1 Given step1 When step2 Then step3
@RegressionTest @test @all Scenario: Test scenario1 Given step11 When step22 Then step33
However the reports are generated for ANDing for the above example ANDing :Tags which are passed in separate quotes are ANDed
Example tags={"@test","@all"} --- Reports are recorded for this tags expression
Version Used