Allure2 report do NOT support for cross browser testing

[dilshan5]

[//]: # ( . Note: for support questions, please use Stackoverflow or Gitter. . This repository's issues are reserved for feature requests and bug reports. . . In case of any problems with Allure Jenkins plugin please use the following repository . to create an issue: https://github.com/jenkinsci/allure-plugin/issues . . Make sure you have a clear name for your issue. The name should start with a capital . letter and no dot is required in the end of the sentence. An example of good issue names: . . - The report is broken in IE11 . - Add an ability to disable default plugins . - Support emoji in test descriptions )

I'm submitting a ...

• [ x] bug report

What is the current behavior?

I use allure report for cross browser testing. Even though the all tests execute in different browsers at the same time but the the report shows only one browser. I used 'AllureCucumber4Jvm'

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

https://github.com/dilshan5/ttaf-ui-intive Sample code is uploaded

What is the expected behavior?

Should display test results for all the browsers which i have executed

What is the motivation / use case for changing the behavior?

Allure support for cross browser testing

Please tell us about your environment:

    <dependency>
        <groupId>io.qameta.allure</groupId>
        <artifactId>allure-cucumber4-jvm</artifactId>
        <version>2.12.1</version>
    </dependency>

Other information

 <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.20</version>
            <configuration>
                <argLine>
                    -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                    -Dcucumber.options="--plugin io.qameta.allure.cucumber4jvm.AllureCucumber4Jvm"
                </argLine>
            </configuration>
            <dependencies>
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjweaver</artifactId>
                    <version>${aspectj.version}</version>
                </dependency>
            </dependencies>
        </plugin>

[//]: # ( . e.g. detailed explanation, stacktraces, related issues, suggestions . how to fix, links for us to have more context, eg. Stackoverflow, Gitter etc )

[dilshan5]

Hi, Is there any update on this?

[dilshan5]

attached allure-results folder allure-results.zip

[auto-qa-git]

Hello, I am trying to do the same thing: See the test reports for each browser using Allure. Any solutions?

[anshuljayn]

Hi, I am facing the same issue. The last execution overrides the previous execution results. Is there any solution. This is preventing me to execute the multi-browser test (at-least from the reporting point of view)

[motivatedmind]

same thing with io.qameta.allure.cucumber5jvm.AllureCucumber5Jvm. No cross browser support yet

[vnpt73]

Hello is there any update? I'm also facing the same issue

[Davidvasantharaj]

Hi, any update on the cross browser issue ? I'm also facing the same issue.

[itkhanz]

Hi, Is there any solution to this issue yet? I am facing the same issue of cross-browser parallel testing with Cucumber where if the scenarios are run in parallel on multiple browsers then the tests are put under Retries tab. I am running my tests with following dependecies:

• Selenium Java 4.6
• Cucumber-TestNG 7.9
• TestNG 7.9
• allure-cucumber7-jvm 2.20.1

Below is how my Test suite looks like

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Smoke Test Suite" thread-count="10" parallel="tests">
    <listeners>
        <listener class-name="framework.listeners.ScenariosParallelTransformer"/>
    </listeners>
    <test  name="Chrome Browser Test">
        <parameter name="browser" value="chrome"/>
        <classes>
            <class name="framework.runners.BaseTestNGRunnerTest"/>
        </classes>
    </test> <!-- Test -->

    <test  name="Firefox Browser Test">
        <parameter name="browser" value="firefox"/>
        <classes>
            <class name="framework.runners.BaseTestNGRunnerTest"/>
        </classes>
    </test> <!-- Test -->
</suite> <!-- Suite -->

As you can see that I am runnin the same BaseTestNGRunnerTeston chrome and firefox in parallel. What actually happens is that if the scenario and/or feature name is same, then the tests are put under Retries Tab in the Test Report for the corresponding scenario, instead of showing it as separate test. In the below pictures you can see that it shows two tests, one as the main test and other as Retry. To differentiate between browsers, I added logs in Hooks.

The Timeline clearly shows the 6 tests running in parallel

but the dashboard and other tabs only shows 3 tests, and mark the remaining 3 tests as Retries.

Expected Behavior: Each Test should be reported as separate test in cross browser parallel testing even if the scenario/feature name is same

Actual Behavior: Tests with same scenario/feature in cross browser parallel testing are shown as same test and the remaining executions of the scenario are put under Retries.

There is also a question for this on Stackoveflow https://stackoverflow.com/questions/58417607/allure-cucumber-jvm-report-for-cross-browser-and-parallel-execution

[DilipMeghwal]

Any work-around or solution for this issue?

[nickator]

Any update on this issue? Quite an old issue and a basic requirement to display results based on the web browser.. Serenity uses "tags" to do this. But no matter what I've tried, like changing the report name based on the web browser, adding dynamic epics, suits, etc... it always adds the test under "Retries" and doesn't display it separately.

[nickator]

workaround: change historyId, but keep there some logic to get everytime same ID + browser name to see history for the test case e.g:

@Story("Your Story Name")
 @Test
  public void loginStandardUser() {
        updateTestNameForAllure(browserPrefix() + "YourStoryName");
        standardUserActions.loginAction();
  }

 private void updateTestNameAndHistoryIdForAllure(String newName) {
        Allure.getLifecycle().updateTestCase(testResult -> testResult.setName(newName));
        String customHistoryId = Utils.generateConsistentHistoryId(newName, getBrowserName());
        Allure.getLifecycle().updateTestCase(testResult -> testResult.setHistoryId(customUUID));
}

privat String generateConsistentHistoryId(String newName, String browserName) {
        String baseString = newName + browserName;
        return UUID.nameUUIDFromBytes(baseString.getBytes()).toString();
  }

[standbyoneself]

workaround: change historyId, but keep there some logic to get everytime same ID + browser name to see history for the test case e.g:

@Story("Your Story Name")
@Test
 public void loginStandardUser() {
       updateTestNameForAllure(browserPrefix() + "YourStoryName");
       standardUserActions.loginAction();
 }

private void updateTestNameAndHistoryIdForAllure(String newName) {
       Allure.getLifecycle().updateTestCase(testResult -> testResult.setName(newName));
       String customHistoryId = Utils.generateConsistentHistoryId(newName, getBrowserName());
       Allure.getLifecycle().updateTestCase(testResult -> testResult.setHistoryId(customUUID));
}

privat String generateConsistentHistoryId(String newName, String browserName) {
       String baseString = newName + browserName;
       return UUID.nameUUIDFromBytes(baseString.getBytes()).toString();
 }

Thanks, it works 🕺