Getting test cases from TestNg suite and test cases from feature file in allure report when executing the test cases in parallel using Cucumber7, Allure-gradle plugin - 2.11.2, gradle 8.0.2, Java11

[Nishant-omar]

Background :- I am working on cucumber framework using gradle and executing in java 11 environment. I have added all latest dependencies in build.gradle file.

Encountering issue - After execution of the test cases in sequential/parallel, when i am generating the allure reprt . In the allure report , i am getting duplicate entry of the same test case, one is from the feature files(BDD behaviour) and other is from the testNG xml file.

But my expectation is to view only the test cases from Feature file(BDD behaviour) , do not want to see the entry from the testng xml.

I have tried to get this resolved by removing the plugin ( id "io.qameta.allure" version "2.11.2") from build.gradle file and that was working fine when i am executing the test cases in sequential .

But when i am executing the test cases in-parallel then the screenshots are only attached with one test case and no screenshot in other test cases and getting below error. "[testng-test=ui test-2] error io.qameta.allure.allurelifecycle - could not add attachment: no test is running"

Summary - Allure report is looking fine for all the test cases when i am executing my test cases in sequential after removing the plugin ("io.qameta.allure" version "2.11.2") from build.gradle but when i am executing the same test cases in parallel ,the screenshot is getting attached with single test case and for the rest of the test case there is no screenshot.

Please refer the below build.gradle, gradle properties, actual allure screenshot before removing the plugin in-parallel execution and expected allure screenshot before removing the plugin in-parallel execution

### build.gradle

plugins { id 'java-library' id "io.qameta.allure" version "2.11.2" }

allure { version = "2.21.0" }

repositories { mavenCentral() }

//allure { // adapter { // autoconfigureListeners.set(true) // aspectjWeaver.set(true) // frameworks { // // Note: every time you mention an adapter, it is added to the classpath, // // so refrain from mentioning unused adapters here // junit5 { // // Disable allure-junit5 default test listeners // autoconfigureListeners.set(false) // } // testng { // // Disable allure-testng default test listeners // autoconfigureListeners.set(false) // } // spock // cucumber7Jvm // } // } //}

dependencies {

//library-reporting-allure
implementation group: 'io.qameta.allure', name: 'allure-cucumber7-jvm', version: '2.21.0'
runtimeOnly group: 'org.aspectj', name: 'aspectjweaver', version: '1.9.5'
implementation group: 'ru.yandex.qatools.ashot', name: 'ashot', version: '1.5.4'
implementation group: 'io.qameta.allure', name: 'allure-plugin-api', version: '2.21.0'

// implementation group: 'io.qameta.allure', name: 'allure-testng', version: '2.15.0'

// library - cucumber
implementation group: 'io.cucumber', name: 'cucumber-java', version: '7.11.2'
testImplementation group: 'io.cucumber', name: 'cucumber-junit', version: '7.11.2'
testImplementation group: 'io.cucumber', name: 'cucumber-picocontainer', version: '7.11.2'
implementation group: 'io.cucumber', name: 'cucumber-testng', version: '7.11.2'
implementation group: 'io.cucumber', name: 'cucumber-java8', version: '7.11.2'

//library-selenium implementation group: 'org.seleniumhq.selenium', name: 'selenium-java', version: '4.8.3'

}

configurations { cucumberRuntime { extendsFrom testImplementation } }

def basedir = 'src/test/resources/testsuites/' def wip = project.hasProperty('cukes.testsuite') ? basedir + project.property('cukes.testsuite') + '.xml' : basedir + 'test' + '.xml'

task wip(type: Test) { useTestNG() { println 'testsuite=' + wip suites wip testLogging.showStandardStreams = true } }

### Gradle-wrapper.properties file distributionUrl=https\://services.gradle.org/distributions/gradle-8.0.2-bin.zip distributionBase=GRADLE_USER_HOME distributionPath=wrapper/dists zipStorePath=wrapper/dists zipStoreBase=GRADLE_USER_HOME

### Expected Allure report in sequential and parallel both type of executions (In below report , i am not getting any test case entry from testng xml. Test cases are only showing from feature files)

Actual Allure report in sequential and parallel before removing the allure plugin from build.gradle. (In below allure screenshot, test cases enteries are showing from testng.xml and feature file)

### MyRunnerClass.java package runners;

@CucumberOptions( // strict = true, features = {"classpath:features/"}, glue = {"steps", "hooks", "uk.ndc.csa.utilities.selenium"}, snippets = CucumberOptions.SnippetType.CAMELCASE, dryRun = false, plugin = {"io.qameta.allure.cucumber7jvm.AllureCucumber7Jvm"} // "com.aventstack.extentreports.cucumber.adapter.ExtentCucumberAdapter:"} )

public abstract class BaseWebStoreTest {

public TestNGCucumberRunner testNGCucumberRunner;
/**
 * Setup before anything runs
 */
@BeforeSuite
public static void reportEnv(){
    TestOrderWriter.getInstance().setupFolder();
}

public abstract test1 gettest1() throws InvalidKeyException;

@Test(dataProvider = "objects")
public void runcukes(Map<String, String> map,PickleWrapper pickleWrapper,FeatureWrapper featureWrapper) throws IOException, InvalidKeyException {
    Boolean selenium = Boolean.parseBoolean(System.getProperty("cukes.selenium", "false"));
    ThreadContext.getInstance().setThreadContext(selenium, map);
    ThreadContext.getInstance().getEnvironmentProps().addProperty("webstore", gettest1());
    testNGCucumberRunner.runScenario(pickleWrapper.getPickle());
}

@DataProvider
public Object[][] scenario(PickleWrapper pickleWrapper,FeatureWrapper featureWrapper){
    testNGCucumberRunner=new TestNGCucumberRunner(this.getClass());
    return testNGCucumberRunner.provideScenarios();
}

/**
 * Returns the list objects providing browser combinations and scenarios.
 *
 * @return the collection of objects.
 */
@DataProvider(name = "objects")
public Object[][] getObject() throws Exception {
    PickleWrapper pickleWrapper=null;
    FeatureWrapper featureWrapper=null;
    return combine(getParams(),  scenario(pickleWrapper, featureWrapper));
}

/**
 * Returns the browser combinations and scenarios.
 *
 * @return the collection of browser combinations and scenarios
 */
public static Object[][] combine(Object[][] a1, Object[][] a2){
    List<Object[]> objectCodesList = new LinkedList<Object[]>();
    for(Object[] o : a1){
        for(Object[] o2 : a2){
            objectCodesList.add(concatAll(o, o2));
        }
    }
    return objectCodesList.toArray(new Object[0][0]);
}

@SafeVarargs
public static <T> T[] concatAll(T[] first, T[]... rest) {
    //calculate the total length of the final object array after the concat
    int totalLength = first.length;
    for (T[] array : rest) {
        totalLength += array.length;
    }
    //copy the first array to result array and then copy each array completely to result
    T[] result = Arrays.copyOf(first, totalLength);
    int offset = first.length;
    for (T[] array : rest) {
        System.arraycopy(array, 0, result, offset, array.length);
        offset += array.length;
    }
    return result;
}

/**
 * save Order Index
 * @throws IOException
 */
@AfterSuite
public void saveOrderIndex() throws IOException {
    TestOrderWriter.getInstance().saveIndex();
}

} @baev , Please help on this.

[baev]

fixed by https://github.com/allure-framework/allure-java/pull/947 . Note, you need to specify group cucumber for runcukes method or use build-in io.cucumber.testng.AbstractTestNGCucumberTests

[rogeriof-zolly]

@baev hey! Where exactly would I need to use the built-in AbstractTestNGCucumberTests? Only by extending it in my runner class? Or do I need to specify it in another config?