prashant-ramcharan / courgette-jvm

Multiprocess | Parallel Cucumber-JVM | Parallelize your Java Cucumber tests on a feature level or on a scenario level.
MIT License
133 stars 39 forks source link

Courgette Reporter shows testname with "2" appended to it #94

Closed bill2605 closed 5 years ago

bill2605 commented 5 years ago

When running my tests, the report xml generated always appends "2" after each test name. Digging through the courgrette files generate in var folders, the test run.json shows:

      ],
        "line": 7,
        "name": "Admin can create a new Channel",
        "description": "",
        "id": "channels;admin-can-create-a-new-channel",
        "after": [

the test run. xml will show:

<testcase classname="Channels" name="Admin can create a new Channel 2" time="37.828686">

while the testrun.html/report.js will show:

formatter.scenario({
  "name": "Admin can create a new Channel",
  "description": "",
  "keyword": "Scenario",
  "tags": [

I am currently failing at trying to figure out where and why the test case name in the xml files add 2 to the test name. Could you shed some light on the above?

courgette runner config:

@RunWith(Courgette.class)
@CourgetteOptions(
        threads = 2,
        runLevel = CourgetteRunLevel.SCENARIO,
        rerunFailedScenarios = false,
        showTestOutput = true,
        cucumberOptions = @CucumberOptions(
                features = { "src/test/resources/features" },
                glue = { "com.workjam.stepdefinition" },
                plugin = {
                        "pretty:target/cucumber-report/cucumber-pretty.txt",
                        "json:target/cucumber-report/cucumber.json",
                        "html:target/cucumber-report/cucumber.html",
                        "junit:target/cucumber-report/cucumber.xml",
                },
                tags = { "@RestReady" }
        )
)
prashant-ramcharan commented 5 years ago

This is the way Cucumber is currently designed.

Cucumber appends the number to the testcase name. This is determined by the number of junit plugins specified in the CucumberOptions.

If you were to use the standard Cucumber runner with multiple junit plugins you will notice this.

For example, this runner will produce 2 junit xml reports and you will notice the "2" appended to the testcase name in _anothercucumber.xml

@RunWith(Cucumber.class)
@CucumberOptions(
        features = { "src/test/resources/features" },
        glue = { "com.workjam.stepdefinition" },
        plugin = {
                "junit:target/cucumber-report/cucumber.xml",
                "junit:target/cucumber-report/another_cucumber.xml",
            },
        tags = { "@RestReady" }
)

Courgette automatically adds 2 junit plugins to the RuntimeOptions before it starts execution so this is why the "2" is appended.

I will look into few options to have only 1 junit plugin added before execution.

bill2605 commented 5 years ago

If i understand this correctly, should i remove any Junit plugin from my runner, i should still see a cucumber.xml generated in the target folder correct? and should that be the case then would the naming issue still occur given that Courgette automatically adds 2 junit plugins to the RuntimeOptions ?

prashant-ramcharan commented 5 years ago

There’s nothing for you to do on your side. Your runner should look exactly the way it is right now.

I will have to make a code change in Courgette to allow only 1 multithreaded junit plugin in the runtime options before the tests execute.

Hope that makes sense.

prashant-ramcharan commented 5 years ago

Can you try with this version?

repositories {
    jcenter()
}

dependencies {
    testCompile 'io.github.prashant-ramcharan:courgette-jvm:3.0.2-snapshot'
}
bill2605 commented 5 years ago

ran on my pipeline and it seems like the issue is no longer occurring :)

prashant-ramcharan commented 5 years ago

Great :thumbsup:

bill2605 commented 5 years ago

Should i close this issue?

prashant-ramcharan commented 5 years ago

That's fine, will keep it open until it's merged into master.

Thanks for reporting.

prashant-ramcharan commented 5 years ago

This is now released in 3.1.0