No Aggregate Report after Failed Test

[abdulsamadf]

Hi!

After a test fails, the aggregate report doesn't generate. I have to run mvn serenity:aggregate afterwards to get this. I've also tried "mvn verify -DtestFailureIgnore=true" but had the same result.

POM - https://github.com/abdulsamadf/Innoved-Test-Serenity/blob/master/innoved-test-framework/innoved_test/pom.xml

MVN Log attached. maven_debug.txt

[YamStranger]

Hello @abdulsamadf, thanks for this issue.

I tested maven plugin using our project: https://github.com/serenity-bdd/serenity-demos/blob/master/junit-webtests/pom.xml, and all works fine.

it seems that something wrong with your pom and not with our maven-plugin, please try to use our demos project or serenity-test-project as example to make your build. Hope all will be fine

If you feel that this issue not about this plugin or you find solution - please close this issue.

Thanks

[abdulsamadf]

I tried running junit-webtests using "mvn clean verify' i get the same issue. (Using POM included with the project).

My maven log: mvn_log2.txt

[abdulsamadf]

I got my project working using POM (https://github.com/abdulsamadf/Innoved-Test-Serenity/blob/master/innoved-test-framework/innoved_test/pom.xml) -

I was missing: <executions> <execution> <goals> <goal>integration-test</goal> <goal>verify</goal> </goals> </execution> </executions>

But i still can't get your junit-website project to run.

[YamStranger]

Hello @abdulsamadf.

In your first comment version 1.1.24 was used:

[DEBUG] Goal: net.serenity-bdd.maven.plugins:serenity-maven-plugin:1.1.24:aggregate (serenity-reports)

In build of junit-webtests: if seems that old version of maven plugin is used:

line 1816: [INFO] --- serenity-maven-plugin:1.1.9:aggregate (serenity-reports) @ serenity-junit-webtests ---

it was released near September: 1.1.9 current version of serenity-maven-plugin is 1.1.25-rc.2 also old version of serenity-core was used:

net.serenity-bdd:serenity-core:1.0.65

@abdulsamadf please use last versions of serenity-core and mvn plugin. Also to generate aggregate reports mvn should not fail by executing tests (same for gradle), so pom should continue configuration for this: example

To make it possible some config should be added

<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <configuration>
          <testFailureIgnore>true</testFailureIgnore>
        </configuration>
      </plugin>

I updated demos project for you: https://github.com/serenity-bdd/serenity-demos/pull/16/files , now it will work out of the box.

So, this is not issue of serenity-maven-plugin but errors in configuration of mvn project build.

If you feel that issue already resolved and you find solution - please close this issue!

Thanks a lot!!!

[Moustakas42]

Is this really closed? If a test fails by timeout why is it considered a build fail? (Using 1.1.8 and testFailureIgnore)

[wakaleo]

Do you have a sample project to illustrate the situation you are describing?

[Moustakas42]

Actually I solved it- I added testFailureIgnore under failsafe as well and I got my report :)

[ManuelAbril]

Hi there,

I had the same problem and I added testFailureIgnore as well. However, I have my project on Jenkins and I want to:

• Get a "red status" if any test fails.
• Aggregate serenity-maven-plugin in order to generate HTML report.

Is that possible?

[wakaleo]

You could run mvn serenity:check after the report generation to force a build failure in Jenkins.

[ManuelAbril]

Thank you so much for your fast answer.

[ManuelAbril]

@wakaleo I was checking the command mvn serenity:check and it works fine. However, it is always failing because: [ERROR] Failed to execute goal net.serenity-bdd.maven.plugins:serenity-maven-plugin:1.1.36:check (default-cli) on project lowi-test: The parameters 'outputDirectoryPath' for goal net.serenity-bdd.maven.plugins:serenity-maven-plugin:1.1.36:check are missing or invalid -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginParameterException

I realized there is an outputDirectoryPath attribute you could add to the plugin. Which is the value we have to use?

[wakaleo]

You shouldn't need to configure the outputDirectoryPath property (it should come from the pom file) - could you post your pom.xml file?

[ManuelAbril]

Of course I can. I explain you what I want to achieve first. I have a CI process in Jenkins and one of the jobs is a SerenityBDD project. It executes after developers build and deploy the application. The main goal is to run a couple of integration tests and get a "green/red" status.

I run it with: mvn clean verify -Ppre-smoke-firefox serenity:check

Here you are my pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.paradigma.lowi</groupId>
    <artifactId>lowi-test</artifactId>
    <version>1.0</version>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <!-- Package versions -->
        <serenity.core.version>1.1.36</serenity.core.version>
        <serenity.maven.plugin.version>1.1.36</serenity.maven.plugin.version>
        <serenity.jbehave.version>1.13.0</serenity.jbehave.version>
        <serenity.junit.version>1.1.36</serenity.junit.version>
        <selenium.core.version>2.53.1</selenium.core.version>
        <junit.version>4.12</junit.version>
        <maven.failsafe.version>2.19.1</maven.failsafe.version>
        <maven.clean.plugin.version>3.0.0</maven.clean.plugin.version>
        <maven.compiler.plugin.version>3.5.1</maven.compiler.plugin.version>
        <maven.dependency.plugin.version>2.10</maven.dependency.plugin.version>
        <maven.install.plugin.version>2.5.2</maven.install.plugin.version>
        <maven.jar.plugin.version>2.5</maven.jar.plugin.version>
        <maven.surefire.plugin.version>2.18</maven.surefire.plugin.version>
        <apache.commons.csv.version>1.4</apache.commons.csv.version>
        <apache.log4j.version>2.6.2</apache.log4j.version>
        <assertj-core>3.5.1</assertj-core>
        <java.current.version>1.8</java.current.version>
    </properties>

    <dependencies>

        <!-- Serenity -->
        <dependency>
            <groupId>net.serenity-bdd</groupId>
            <artifactId>serenity-core</artifactId>
            <version>${serenity.core.version}</version>
        </dependency>
        <dependency>
            <groupId>net.serenity-bdd</groupId>
            <artifactId>serenity-jbehave</artifactId>
            <version>${serenity.jbehave.version}</version>
        </dependency>
        <dependency>
            <groupId>net.serenity-bdd</groupId>
            <artifactId>serenity-junit</artifactId>
            <version>${serenity.junit.version}</version>
        </dependency>
        <dependency>
            <groupId>net.serenity-bdd</groupId>
            <artifactId>serenity-screenplay</artifactId>
            <version>${serenity.core.version}</version>
        </dependency>

        <!-- Selenium -->
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>${selenium.core.version}</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-firefox-driver</artifactId>
            <version>${selenium.core.version}</version>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-server</artifactId>
            <version>${selenium.core.version}</version>
        </dependency>

        <!-- Junit -->
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>${junit.version}</version>
        </dependency>

        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-csv</artifactId>
            <version>${apache.commons.csv.version}</version>
        </dependency>

        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-api</artifactId>
            <version>${apache.log4j.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>${apache.log4j.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-slf4j-impl</artifactId>
            <version>${apache.log4j.version}</version>
            <scope>runtime</scope>
        </dependency>

        <dependency>
            <groupId>org.assertj</groupId>
            <artifactId>assertj-core</artifactId>
            <version>${assertj-core}</version>
        </dependency>

    </dependencies>

    <build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>${maven.compiler.plugin.version}</version>
                    <configuration>
                        <source>${java.current.version}</source>
                        <target>${java.current.version}</target>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-dependency-plugin</artifactId>
                    <version>${maven.dependency.plugin.version}</version>
                </plugin>
            </plugins>
        </pluginManagement>

        <plugins>
            <plugin>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>${maven.failsafe.version}</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>${maven.surefire.plugin.version}</version>
                <configuration>
                    <argLine>-Xmx1024m</argLine>
                    <useSystemClassLoader>true</useSystemClassLoader>
                    <skipTests>false</skipTests>
                    <testFailureIgnore>true</testFailureIgnore>
                    <systemPropertyVariables>
                        <webdriver.driver>${webdriver.driver}</webdriver.driver>
                        <test.environment>${test.environment}</test.environment>
                        <metafilter>${meta.filter}</metafilter>
                    </systemPropertyVariables>
                </configuration>
            </plugin>
            <plugin>
                <groupId>net.serenity-bdd.maven.plugins</groupId>
                <artifactId>serenity-maven-plugin</artifactId>
                <version>${serenity.maven.plugin.version}</version>
                <executions>
                    <execution>
                        <id>serenity-reports</id>
                        <phase>post-integration-test</phase>
                        <goals>
                            <goal>aggregate</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

    <profiles>
        <!-- Profile naming: environment_testphase_webdriver -->
        <profile>
            <id>pre-smoke-firefox</id>
            <properties>
                <test.environment>pre</test.environment>
                <meta.filter>+phase smoke</meta.filter>
                <webdriver.driver>firefox</webdriver.driver>
            </properties>
        </profile>
        <profile>
            <id>pro-smoke-firefox</id>
            <properties>
                <test.environment>pro</test.environment>
                <meta.filter>+phase smoke</meta.filter>
                <webdriver.driver>firefox</webdriver.driver>
            </properties>
        </profile>
    </profiles>
</project>

[ManiKalyani234]

I am running successfully my tests with serenity cucumber in maven project, but when I am trying to convert the same project to gradle, I could not able to run the tests. When I use gradle clean test aggregate, I do see not tests are running. Could you please help me, to provide a good gradle.build file to execute my feature files.

[wakaleo]

@ManiKalyani234 You should post this question on https://serenity-bdd.rocket.chat