devonfw / mrchecker

End to End (E2E) test framework - MrChecker
33 stars 38 forks source link

Mrchecker with Cucumber no steps are shown in Allure Report #215

Closed mcpabst closed 12 months ago

mcpabst commented 4 years ago

image

After updating/adapting your example pom.xml the "Steps" from cucumber are missing in allure report

`<?xml version="1.0" encoding="UTF-8"?> <project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

4.0.0
<artifactId>example-app-under-test</artifactId>
<groupId>com.capgemini.mrchecker</groupId>
<version>${version}</version>
<packaging>jar</packaging>

<name>MrCheckerTestFramework - List of Integration Test for Application Under Test</name>
<description>MrChecker Test Framework is an automated testing framework for functional and non functional tests of web applications, native mobile apps, webservices and database.</description>
<url>https://github.com/devonfw/devonfw-testing</url>

<organization>
    <name>Capgemini - Nearshore Test Center - Poland</name>
    <url>http://nsc.capgemini.com/nearshoretestcenter</url>
</organization>

<developers>
    <developer>
        <name>Lukasz Stefaniszyn</name>
        <email>lukasz.stefaniszyn@capgemini.com</email>
        <organization>Capgemini - Nearshore Test Center - Poland</organization>
        <organizationUrl>http://nsc.capgemini.com/nearshoretestcenter</organizationUrl>
    </developer>
</developers>

<licenses>
    <license>
        <name>The Apache License, Version 2.0</name>
        <url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
    </license>
</licenses>

<scm>
    <connection>scm:git:git://github.com/devonfw/devonfw-testing.git</connection>
    <developerConnection>scm:git:ssh://github.com:devonfw/devonfw-testing.git</developerConnection>
    <url>http://github.com/devonfw/devonfw-testing/tree/develop</url>
</scm>

<repositories>
    <repository>
        <id>maven-central</id>
        <url>http://repo1.maven.org/maven2</url>
    </repository>
</repositories>

<distributionManagement>
    <!-- Repository used as default deploy phase. -->
    <repository>
        <id>Local repository</id>
        <url>file://${env.BASE_PATH}/usr/share/maven2</url>
        <!-- <url>file://${env.BASE_PATH}/m2/repository</url> -->
    </repository>
</distributionManagement>

<properties>
    <version>2.6.0</version>

    <project.build.sourceEncoding>UTF8</project.build.sourceEncoding>

    <!-- Allure test framework modules -->
    <mrchecker-core-module.version>4.12.1.1</mrchecker-core-module.version>
    <mrchecker-selenium-module.version>3.8.2.1</mrchecker-selenium-module.version>
    <mrchecker-security-module.version>1.2.0</mrchecker-security-module.version>
    <mrchecker-webapi-module.version>1.2.0</mrchecker-webapi-module.version>

    <!-- Java settings -->
    <java.compiler.version>1.8</java.compiler.version>

    <!-- Tests executors -->
    <maven-surefire-plugin.version>2.19.1</maven-surefire-plugin.version>
    <surefire.provider.version>1.0.0-M2</surefire.provider.version>
    <thread.count>8</thread.count>
    <junit-toolbox.version>2.4</junit-toolbox.version>

    <!-- Maven report generator -->
    <maven-site-plugin.version>3.3</maven-site-plugin.version>
    <maven-project-info-reports-plugin.version>2.9</maven-project-info-reports-plugin.version>

    <!-- Allure reporting -->
    <allure-cli-runner.version>2.6.0</allure-cli-runner.version> <!-- Url to download Allure report CLI https://bintray.com/qameta/generic/allure2 -->
    <allure.report.directory>${pom.basedir}/target/</allure.report.directory>
    <report-directory>${pom.basedir}/test-output/</report-directory>
    <aspectj.version>1.8.10</aspectj.version>

    <!-- Cucumber version -->
    <cucumber.jvm.parallel.version>5.0.0</cucumber.jvm.parallel.version>
    <cucumber-reporting.version>3.7.0</cucumber-reporting.version>
    <cucumber.tag>@all</cucumber.tag>

    <!-- Test tools -->
    <hamcrest.version>1.3</hamcrest.version>
    <junit.version>4.12</junit.version>
    <mockito.version>1.8.5</mockito.version>
    <joda-time.version>2.9.9</joda-time.version>
    <jsoup.version>1.11.2</jsoup.version>
    <commons-csv.version>1.1</commons-csv.version>
    <JUnitParams.version>1.0.4</JUnitParams.version>

    <!-- Dependency Injection -->
    <guice.version>4.1.0</guice.version>

    <!-- Static analyze settings -->
    <formatter-maven-plugin.version>2.7.1</formatter-maven-plugin.version>
    <style_format.directory>${pom.basedir}/../resources/style_format</style_format.directory>

</properties>

<dependencies>
    <!-- Dependency to other MrChecker Test Framework modules -->
    <!-- Test core module -->
    <dependency>
        <groupId>${project.groupId}</groupId>
        <artifactId>mrchecker-core-module</artifactId>
        <version>${mrchecker-core-module.version}</version>
    </dependency>

    <!-- Selenium module -->
    <dependency>
        <groupId>${project.groupId}</groupId>
        <artifactId>mrchecker-selenium-module</artifactId>
        <version>${mrchecker-selenium-module.version}</version>
    </dependency>

    <!-- Security module -->
    <dependency>
        <groupId>${project.groupId}</groupId>
        <artifactId>mrchecker-security-module</artifactId>
        <version>${mrchecker-security-module.version}</version>
    </dependency>

    <!-- WebAPI + Service Virtualization module -->
    <dependency>
        <groupId>${project.groupId}</groupId>
        <artifactId>mrchecker-webapi-module</artifactId>
        <version>${mrchecker-webapi-module.version}</version>
    </dependency>

    <dependency>
        <groupId>org.aspectj</groupId>
        <artifactId>aspectjweaver</artifactId>
        <version>${aspectj.version}</version>
    </dependency>

    <!--This dependency is necessary for JUnit plugin. -->
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>${junit.version}</version>
    </dependency>
    <dependency>
        <groupId>pl.pragmatists</groupId>
        <artifactId>JUnitParams</artifactId>
        <version>${JUnitParams.version}</version>
    </dependency>

    <dependency>
        <groupId>org.junit.platform</groupId>
        <artifactId>junit-platform-surefire-provider</artifactId>
        <version>${surefire.provider.version}</version>
    </dependency>

    <dependency>
        <groupId>org.hamcrest</groupId>
        <artifactId>hamcrest-all</artifactId>
        <version>${hamcrest.version}</version>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>org.mockito</groupId>
        <artifactId>mockito-core</artifactId>
        <version>${mockito.version}</version>
    </dependency>

    <dependency>
        <groupId>joda-time</groupId>
        <artifactId>joda-time</artifactId>
        <version>${joda-time.version}</version>
    </dependency>

    <dependency>
        <groupId>org.apache.commons</groupId>
        <artifactId>commons-csv</artifactId>
        <version>${commons-csv.version}</version>
    </dependency>
    <!--This dependency is necessary for testsuits runned with WildcardPatternSuite -->
    <dependency>
        <groupId>com.googlecode.junit-toolbox</groupId>
        <artifactId>junit-toolbox</artifactId>
        <version>${junit-toolbox.version}</version>
    </dependency>

      <dependency>
        <groupId>org.assertj</groupId>
        <artifactId>assertj-core</artifactId>
        <version>3.11.0</version>
    </dependency>

    <dependency>
        <groupId>org.awaitility</groupId>
        <artifactId>awaitility</artifactId>
        <version>3.0.0</version>
    </dependency>

    <dependency>
        <groupId>com.github.automatedowl</groupId>
        <artifactId>allure-environment-writer</artifactId>
        <version>1.0.0</version>
    </dependency>
</dependencies>

<!-- Build goals/actions -->
<build>
    <pluginManagement>
        <plugins>

            <!-- CLEAN -->
            <plugin>
                <artifactId>maven-clean-plugin</artifactId>
                <version>3.0.0</version>
                <configuration>
                    <filesets>
                        <fileset>
                            <directory>logs</directory>
                            <followSymlinks>false</followSymlinks>
                        </fileset>
                        <fileset>
                            <directory>${report-directory}</directory>
                        </fileset>
                    </filesets>
                    <fileset>
                            <directory>bin/cucumber/</directory>
                            <directory>cucumber/</directory>
                            <includes>
                                <include>**</include>
                            </includes>
                            <followSymlinks>false</followSymlinks>
                        </fileset>
                </configuration>
            </plugin>

            <!-- Static Analysis Plugins -->
            <plugin><!-- mvn java-formatter:format -->
                <groupId>net.revelc.code.formatter</groupId>
                <artifactId>formatter-maven-plugin</artifactId>
                <version>${formatter-maven-plugin.version}</version>
                <configuration>
                    <configFile>${style_format.directory}/eclipse_format_code_standards.xml</configFile>
                    <lineEnding>LF</lineEnding>
                    <encoding>${project.build.sourceEncoding}</encoding>
                </configuration>
                <executions>
                    <execution>
                        <id></id>
                        <phase>verify</phase>
                        <goals>
                            <goal>validate</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

            <!-- Maven report generator -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-site-plugin</artifactId>
                <version>${maven-site-plugin.version}</version>
            </plugin>

        </plugins>
    </pluginManagement>
</build>

<profiles>
    <!-- RUN CLASSIC JUNIT TESTS -->
    <profile>
        <id>default</id>
        <activation>
            <activeByDefault>true</activeByDefault>
        </activation>
        <build>
            <plugins>

                <!-- COMPILE -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.6.1</version>

                    <configuration>
                        <excludes>
                            <exclude>**/cucumber/*.java</exclude>
                        </excludes>
                        <compilerArgument>-Xlint:deprecation</compilerArgument>
                        <showWarnings>false</showWarnings>
                        <showDeprecation>true</showDeprecation>
                        <source>${java.compiler.version}</source>
                        <target>${java.compiler.version}</target>
                        <encoding>UTF-8</encoding>
                        <testSource>${java.compiler.version}</testSource>
                        <testTarget>${java.compiler.version}</testTarget>
                    </configuration>
                </plugin>

                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>${maven-surefire-plugin.version}</version>
                    <configuration>
                          <skipTests>false</skipTests>
                        <forkCount>${thread.count}</forkCount>
                        <reuseForks>false</reuseForks>
                        <testFailureIgnore>true</testFailureIgnore>
                        <failIfNoTests>false</failIfNoTests>
                        <includes>
                            <include>**/*IT.class</include>
                        </includes>
                        <excludes>
                            <exclude>**/RawTest.*</exclude>
                            <exclude>**/OutboxDocumentTest.*</exclude>
                        </excludes>
                        <argLine>
                            -javaagent:${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar
                            -Xmx1024m -Xms1024m -XX:MaxPermSize=512m
                            -Djdk.net.URLClassPath.disableClassPathURLCheck=true
                        </argLine>
                        <properties>
                            <property>
                                <name>listener</name>
                                <value>io.qameta.allure.junit4.AllureJunit4</value>
                            </property>

                        </properties>
                        <systemProperties>
                            <property>
                                <name>allure.results.directory</name>
                                <value>${allure.report.directory}/allure-results</value>
                            </property>
                        </systemProperties>
                    </configuration>
                    <dependencies>
                        <dependency>
                            <groupId>org.aspectj</groupId>
                            <artifactId>aspectjweaver</artifactId>
                            <version>${aspectj.version}</version>
                        </dependency>
                    </dependencies>
                </plugin>
                <plugin>
                    <groupId>io.qameta.allure</groupId>
                    <artifactId>allure-maven</artifactId>
                    <version>2.10.0</version>
                    <configuration>
                        <!-- Url to download Allure report CLI https://bintray.com/qameta/generic/allure2 -->
                        <reportVersion>${allure-cli-runner.version}</reportVersion>
                        <resultsDirectory>${allure.report.directory}/allure-results</resultsDirectory>
                        <reportDirectory>${allure.report.directory}/site/allure-report</reportDirectory>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </profile>

    <!--'parallelPlugin' profile is executing tests in parallel - with
        cucumber-jvm-parallel-plugin -->
    <profile>
        <id>cucumber-parallel</id>
        <properties>
            <include.runners>**/Parallel*IT.class</include.runners>
        </properties>
        <build>
            <plugins>
                <plugin>
                    <groupId>com.github.temyers</groupId>
                    <artifactId>cucumber-jvm-parallel-plugin</artifactId>
                    <version>${cucumber.jvm.parallel.version}</version>
                    <executions>
                        <execution>
                            <id>generateRunners</id>
                             <phase>generate-test-sources</phase>
                             <!--<phase>validate</phase>-->

                            <goals>
                                <goal>generateRunners</goal>
                            </goals>
                            <configuration>
                                <!-- Mandatory -->
                                <!-- comma separated list of package
                                    names to scan for glue code -->
                                <glue>
                                 <!--  <package>com.capgemini.mrchecker.cucumber.stepdefs</package>
                                    <package>com.capgemini.mrchecker.cucumber.hooks</package>-->
                                    <package>com.capgemini.b.b.steps</package>
                                </glue>
                                <!-- These are optional, with the default
                                    values -->
                                <!-- Where to output the generated tests -->
                                <outputDirectory>${project.build.directory}/generated-test-sources/cucumber</outputDirectory>
                                <!-- The directory, which must be in
                                    the root of the runtime classpath,
                                    containing your feature files. -->
                                <featuresDirectory>src/main/resources/features</featuresDirectory>
                                <!-- Directory where the cucumber report
                                    files shall be written -->
                                <cucumberOutputDir>${project.build.directory}/cucumber</cucumberOutputDir>
                                <!-- List of cucumber plugins. When none
                                    are provided the json formatter is
                                    used. For more
                                    advanced usage see section about
                                    configuring cucumber plugins -->
                                <plugins>
                                    <plugin>
                                        <!--The available options are
                                            junit, testng, html, pretty,
                                            json, usage and rerun -->
                                        <name>json</name>
                                    </plugin>
                                    <plugin>
                                        <!--The available options are
                                            junit, testng, html, pretty,
                                            json, usage and rerun -->
                                        <name>html</name>
                                        <outputDirectory>${project.build.directory}/cucumber-html-report</outputDirectory>
                                    </plugin>
                                    <plugin>
                                        <name>pretty</name>
                                    </plugin>
                                    <plugin>
                                        <name>junit</name>
                                        <outputDirectory>${project.build.directory}/junitreports</outputDirectory>
                                    </plugin>
                                </plugins>
                                <!-- CucumberOptions.strict property -->
                                <strict>true</strict>
                                <!-- CucumberOptions.monochrome property -->
                                <monochrome>false</monochrome>
                                <!--The tags to run, maps to CucumberOptions.tags
                                    property -->
                                <tags>
                                    <!-- Logical OR -Dtag=“@SoapUI,@Functional" -->
                                    <!-- Logical AND -Dtag=“@SoapUI","@Functional" -->
                                    <tag>${cucumber.tag}</tag>
                                    <tag>~@ignored</tag>
                                </tags>
                                <!-- Generate TestNG runners instead
                                    of JUnit ones. -->
                                <useTestNG>false</useTestNG>
                                <!-- The naming scheme to use for the
                                    generated test classes. One of 'simple'
                                    or 'feature-title' -->
                                <namingScheme>feature-title</namingScheme>
                                <!-- The class naming pattern to use.
                                    Only required/used if naming scheme
                                    is 'pattern'. -->
                                <namingPattern>Parallel{c}IT</namingPattern>
                                <!-- One of [SCENARIO, FEATURE]. SCENARIO
                                    generates one runner per scenario.
                                    FEATURE generates a runner per feature. -->
                                <parallelScheme>FEATURE</parallelScheme>
                                <!-- Specify a custom template for the
                                    generated sources (this is a path
                                    relative to the project base directory) -->
                                <customVmTemplate>src/resources/custom-template.vm</customVmTemplate>
                                <!-- Specify a custom package name for
                                    generated sources. Default is no
                                    package. -->
                                <outputDirectory>${basedir}/src/test/java</outputDirectory>
                               <packageName>com.capgemini.b.b.tests.autogenerated_runners</packageName>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>

                <!-- COMPILE CUCUMBER FILES -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.6.1</version>

                    <configuration>
                        <compilerArgument>-Xlint:deprecation</compilerArgument>
                        <showWarnings>false</showWarnings>
                        <showDeprecation>true</showDeprecation>
                        <source>${java.compiler.version}</source>
                        <target>${java.compiler.version}</target>
                        <encoding>UTF-8</encoding>
                        <testSource>${java.compiler.version}</testSource>
                        <testTarget>${java.compiler.version}</testTarget>
                    </configuration>
                </plugin>

                <!-- <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.6.1</version>
                    <executions>
                    <execution>
                    <id>cucumber-test-compile</id>
                    <phase>test-compile</phase>
                    <goals>
                    <goal>testCompile</goal>
                    </goals>
                    </execution>
                    </executions>
                    <configuration>
                    <compilerArgument>-Xlint:deprecation</compilerArgument>
                    <showWarnings>false</showWarnings>
                    <showDeprecation>true</showDeprecation>
                    <source>${java.compiler.version}</source>
                    <target>${java.compiler.version}</target>
                    <encoding>UTF-8</encoding>
                    <testSource>${java.compiler.version}</testSource>
                    <testTarget>${java.compiler.version}</testTarget>
                    </configuration>
                    </plugin> -->

                <!-- EXECUTE TESTS -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>${maven-surefire-plugin.version}</version>
                    <executions>
                        <execution>
                            <phase>test</phase>
                        </execution>
                    </executions>
                    <configuration>
                        <skipTests>false</skipTests>
                        <forkCount>${thread.count}</forkCount>
                        <reuseForks>false</reuseForks>
                        <!--  <includes>**/Parallel*IT.*</includes>-->
                        <includes>
                            <include>**/*.class</include>
                        </includes>
                         <excludes>
                            <exclude>**/RawTest.*</exclude>
                            <exclude>**/OutboxDocumentTest.*</exclude>
                        </excludes>
                        <testClassesDirectory>${project.build.testOutputDirectory}</testClassesDirectory>

                        <testFailureIgnore>true</testFailureIgnore>
                        <failIfNoTests>false</failIfNoTests>
                        <argLine>
                            -javaagent:${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar
                            -Xmx1024m -Xms1024m -XX:MaxPermSize=512m

                        </argLine>
                        <properties>
                            <property>
                                <name>listener</name>
                                <value>io.qameta.allure.junit4.AllureJunit4</value>
                            </property>

                        </properties>
                        <systemProperties>
                            <property>
                                <name>allure.results.directory</name>
                                <value>${allure.report.directory}/allure-results</value>
                            </property>
                        </systemProperties>

                    </configuration>
                    <dependencies>
                        <dependency>
                            <groupId>org.apache.maven.surefire</groupId>
                            <artifactId>surefire-junit4</artifactId>
                            <version>${maven-surefire-plugin.version}</version>
                        </dependency>
                    </dependencies>
                </plugin>

                <!-- GENERATE ALLURE CUCUMBER REPORT -->
                <plugin>
                    <groupId>io.qameta.allure</groupId>
                    <artifactId>allure-maven</artifactId>
                    <configuration>
                        <!-- Url to download Allure report CLI https://bintray.com/qameta/generic/allure2 -->
                        <reportVersion>${allure-cli-runner.version}</reportVersion>
                        <resultsDirectory>${allure.report.directory}/allure-results</resultsDirectory>
                        <reportDirectory>${allure.report.directory}/site/allure-report</reportDirectory>
                    </configuration>
                </plugin>

                <!-- GENERATE CLASSIC CUCUMBER REPORT -->
                <plugin>
                    <groupId>net.masterthought</groupId>
                    <artifactId>maven-cucumber-reporting</artifactId>
                    <version>${cucumber-reporting.version}</version>
                    <executions>
                        <execution>
                            <id>execution</id>
                            <phase>verify</phase>
                            <goals>
                                <goal>generate</goal>
                            </goals>
                            <configuration>
                                <cucumberOutput>${project.build.directory}/cucumber.json</cucumberOutput>
                                <projectName>CucumberReport</projectName>
                                <outputDirectory>${project.build.directory}/cucumber-html-report</outputDirectory>
                                <parallelTesting>true</parallelTesting>
                                <buildNumber>1</buildNumber>
                                <checkBuildResult>false</checkBuildResult>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>

            </plugins>
        </build>
    </profile>

</profiles>

<reporting>
    <plugins>
        <!-- Report project information -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-project-info-reports-plugin</artifactId>
            <version>${maven-project-info-reports-plugin.version}</version>
            <reportSets>
                <reportSet>
                    <reports>
                        <report>scm</report>
                    </reports>
                </reportSet>
            </reportSets>
        </plugin>
    </plugins>
</reporting>

`

baklarzej commented 4 years ago

The issue can likely be solved by using allure-cucumber4-jvm in the cucumber profiles, like here: Allure -> Cucumber JVM

io.qameta.allure.junit4.AllureJunit4 shouldn't be used as a listener for cucumber profiles, because it's designed for JUnit, not Cucumber reporting.

allure-cucumberjvm Adaptor also worked but it's deprecated and for me it strangely caused additional problems when I started using RestAssured in the cucumber steps.

KMariusz commented 12 months ago

It is solved in latest versions of MrChecker and with new special example project (Selenium Cucumber)