Multi-Module JUnit Screenplay Report Aggregation Problems

[terrystrachan]

I am setting up a multi-module maven project to create UI tests for a large browser based application using junit and the screenplay bdd pattern.

Each module runs the tests and reports as expected individually, however, if I try to run the tests at the top level of the project then - ALL of the tests run, but the report doesn't include ALL of the detail for the child modules. I get a report with test details covering all modules, but NOT requirements, features and stories information consolidated

I have created a simple test example as follows;

The "simple" Maven project was created in Eclipse and the module test projects were created with

mvn archetype:generate -Dfilter=screenplay

and modified the tests to be different on each one.

The pom was updated to use serenity version 2.0.81

I have the following in the poms at all levels in the project

<plugin>
  <groupId>net.serenity-bdd.maven.plugins</groupId>
  <artifactId>serenity-maven-plugin</artifactId>
  <version>${serenity.version}</version>
  <executions>
    <execution>
      <id>serenity-reports</id>
      <phase>post-integration-test</phase>
      <goals>
        <goal>aggregate</goal>
      </goals>
    </execution>
  </executions>
</plugin>

Project folder structure is;

UITesting
  pom.xml
  serenity.properties
  demo_one
    src
      test
        java
        resources
          uk / co / test / 
            features
              feature_folder_one
                package-info.java
                StoryClassOne.java
            tasks
            ui
    pom.xml
    serenity.properties
  demo_two
    src
      test
        java
        resources
          uk / co / test / 
            features
              feature_folder_two
                package-info.java
                StoryClassTwo.java
            tasks
            ui
    pom.xml
    serenity.properties

and same for demo 3 and I set the output directory in the each level serenity.properties file serenity.outputDirectory=c:/temp/uireport

I was hoping that the Requirements, Features and Stories tabs would consolidate all of the child module report data. But ONLY the actual tests and tags are consolidated, nothing else is.

Is this possible, or have i missed something?

[wakaleo]

This won't happen automatically - multi-module tests aren't supported in this way.

[terrystrachan]

Ah, ok - thanks...

Any pointers on how I could achieve this? or would I have to build a separate dashboard (webpage) to access the reports in each child module directly?

[terrystrachan]

Interestingly - after running

mvn clean verify

at the top project level and then updating the array in

uk.co.test.features-package-requirements

to include the details which is generated when running the modules individually - and then running

mvn serenity:aggregate

the report then shows the consolidated requirements, features and stories - seemingly with the valid links.

Is it feasible to request that the aggregation phase inserts the requirements.json data instead of overwriting it?

[wakaleo]

Hey Terry, do you have a sample project to reproduce this?

[terrystrachan]

Hey John,

Unfortunately, I canned that example and the working version I currently have is very specific to our company requirements.

I have created a new multi module maven project which mimics what i'm trying to achieve. I've uploaded it to github - here https://github.com/terrystrachan/multi-module-maven-serenity

The serenity.properties at project level have the following line (plus other properties)    serenity.outputDirectory=../../../reports    so that the report output is collated at the top level

The idea was to be able to run the tests individually, or by domain, or as a total set etc... With the reports aggregated to the top level.

Currently, if I run the individual tests (demo_one, demo_two, or demo_three) with mvn clean verify I get the report at the top level as expected.Requirements, features and stories appear to be populated as expected.

If i dont delete the folder, and run each individual test - 1 after the other - i get 3 test scenarios visible and can see the 3 test results (from the overall test results page)but the requirements, features and stories tabs only show the last test ran.

If i run the test at domain level - eg domain_two - by deleting the report folder and running mvn clean verify demo_two and demo_three tests run - and pass :) - the report shows 2 test scenarios and the requirements, features, and stories tabs only show the last test ran.

If I run it at the very top level - the tests run, but the report doesn't contain anything. running mvn serenity:aggregate shows the 3 tests results in the Overall test results tab but nothing in the requirements tab and the features and stories tab are not present. (there were a lot of scrolling errors as the aggregation progressed - however it did cite build success - added Aggregate Errors.txt file to root)

I'm sure i had a previous version of this working when I copied the features-package-requirements.json (example file features-package-requires-example.txt added to root) for each individual test into 1 single file and then ran mvn serenity:aggregate - but that isnt currently working - I'll spend a little more time on that.....

cheers

Terry

[terrystrachan]

Is that our period over then?

On Sat, 12 Feb 2022, 15:10 John Ferguson Smart, @.***> wrote:

Closed #1853 https://github.com/serenity-bdd/serenity-core/issues/1853.

— Reply to this email directly, view it on GitHub https://github.com/serenity-bdd/serenity-core/issues/1853#event-6057178348, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABFROOVQFIPNLYDYUNNGOJLU2Z2ADANCNFSM4JEARBHQ . You are receiving this because you modified the open/close state.Message ID: @.*** com>

[wakaleo]

This was closed automatically, sorry.