Quarkus doen't use @DisplayName for Failsafe reports

Describe the bug Quarkus is not using the @DisplayName configured on the integration tests in the report generated by failsafe report.

Expected behavior when running mvnw verify failsafe:integration-test failsafe:verify

the @DisplayName should appear in the testcase name on the report.

Actual behavior testcase name is currently the method name.

Configuration

POM.xml

<profiles>
        <profile>
            <id>native</id>
            <properties>
                <quarkus.package.type>native</quarkus.package.type>
            </properties>
            <activation>
                <property>
                    <name>native</name>
                </property>
            </activation>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-failsafe-plugin</artifactId>
                        <version>${surefire-plugin.version}</version>
                        <configuration>
                            <statelessTestsetReporter implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5Xml30StatelessReporter">
                                <disable>false</disable>
                                <version>3.0</version>
                                <usePhrasedTestCaseMethodName>true</usePhrasedTestCaseMethodName>
                            </statelessTestsetReporter>
                        </configuration>
                        <executions>
                            <execution>
                                <goals>
                                    <goal>integration-test</goal>
                                    <goal>verify</goal>
                                </goals>
                                <configuration>
                                    <systemPropertyVariables>
                                        <native.image.path>${project.build.directory}/${project.build.finalName}-runner</native.image.path>
                                        <java.util.logging.manager>org.jboss.logmanager.LogManager</java.util.logging.manager>
                                        <maven.home>${maven.home}</maven.home>
                                    </systemPropertyVariables>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>

Example of test java class:

@QuarkusTest
@Tag("integration")
class DescriptionResourceIT extends AuthTestExtension {

    @BeforeAll
    static void giveMeAMapper() {
        final Jsonb jsonb = JsonbBuilder.create();
        ObjectMapper mapper = new ObjectMapper() {
            @Override
            public Object deserialize(ObjectMapperDeserializationContext context) {
                return jsonb.fromJson(context.getDataToDeserialize().asString(), context.getType());
            }

            @Override
            public Object serialize(ObjectMapperSerializationContext context) {
                return jsonb.toJson(context.getObjectToSerialize());
            }
        };
        RestAssured.config = RestAssured.config().objectMapperConfig(
            ObjectMapperConfig.objectMapperConfig().defaultObjectMapper(mapper));
    }

    @Test
    @DisplayName("Testing displayName")
    void assertOrderOfGroups() {
        DescriptionRequest descriptionRequest = validRequest();
        descriptionRequest.setFeatureCodes(List.of("XXXX", "XXXX1", "XXXX2"));
        descriptionRequest.setSelectedDate("2020-12-02");
        authGiven()
            .header("Content-Type", MediaType.APPLICATION_JSON)
            .body(descriptionRequest)
            .when()
            .post("/descriptions/full")
            .then()
            .statusCode(200)
            .body("groups[0].groupNumber", equalTo("1"))
            .body("groups[1].groupNumber", equalTo("2.1"))
            .body("groups[2].groupNumber", equalTo("2.2"))
            .body("groups[3].groupNumber", equalTo("2.3"))
            .body("groups[4].groupNumber", equalTo("11"));
    }

    }

Screenshots

Failsafe report .xml

Environment (please complete the following information): java -version openjdk version "11.0.9.1" 2020-11-04 OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.9.1+1) OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.9.1+1, mixed mode)

Quarkus version is 1.9.1.Final

mvnw --version Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f) Maven home: C:\Users\user.m2\wrapper\dists\apache-maven-3.6.3-bin\39u95cbq5nt4o0nbmqmiptki14\apache-maven-3.6.3 Java version: 11.0.9.1, vendor: AdoptOpenJDK, runtime: C:\Program Files\AdoptOpenJDK\jdk-11.0.9.101-hotspot Default locale: en_US, platform encoding: Cp1252 OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"

Failsafe and Surefire version is 3.0.0-M5

Additional context (Add any other context about the problem here.)

Is this even a Quarkus issue?

I mean do regular (non-Quarkus) tests annotated with @DisplayName show up like you mention in the reports?

Is this even a Quarkus issue?

I mean do regular (non-Quarkus) tests annotated with @DisplayName show up like you mention in the reports?

If I run non-Quarkus tests annotated with @DisplayName I got no problem, this only happens when its a quarkus test.

I am not seeing that. I am seeing @Display name not being used at all in the surefire reports

I just came across some maybe related issue ...

I am not seeing that. I am seeing @Display name not being used at all in the surefire reports

The default maven-surefire-plugin and maven-failsafe-plugin configuration does not use the @DisplayName annotation for the generated reports. However it is possible to configure the plugin accordingly to use the "phrased" names in reports as shown in the first post:

<configuration>
            <statelessTestsetReporter
              implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5Xml30StatelessReporter">
              <disable>false</disable>
              <version>3.0</version>
              <usePhrasedFileName>false</usePhrasedFileName>
              <usePhrasedTestSuiteClassName>true</usePhrasedTestSuiteClassName>
              <usePhrasedTestCaseClassName>true</usePhrasedTestCaseClassName>
              <usePhrasedTestCaseMethodName>true</usePhrasedTestCaseMethodName>
            </statelessTestsetReporter>
          </configuration>

Is this even a Quarkus issue?

This is not really a Quarkus issue imho. However Quarkus uses in almost every test-related tutorial the example of writing a @QuarkusMainTest and then write an integration test for the native build by simply extending the test and annotate it with @QuarkusMainIntegrationTest:

@QuarkusMainTest
public class DemoTest {

  @Test
  public void givenSome_whenAction_theExpectOther() {
    Assertions.assert(other);
  }
}

@QuarkusMainIntegrationTest
public class DemoIT extends DemoTest {}

As soon as somebody annotates the test with @DisplayName he/she might expect this DisplayName to be used in the integration test too. But @DisplayName is not inherited.

Perhaps Quarkus could reduce the irritations by providing a DisplayNameGenerator that inspects super classes for given @DisplayNameannotations and uses them if found for the integration test.

An incomplete example for such a DisplayNameGenerator is:

import java.lang.reflect.Method;

import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.DisplayNameGenerator;

public class InheritedDisplayNameDisplayNameGenerator extends DisplayNameGenerator.Standard {

  @Override
  public String generateDisplayNameForMethod(Class<?> testClass, Method testMethod) {
    if (testMethod.isAnnotationPresent(DisplayName.class)) {
      return super.generateDisplayNameForMethod(testClass, testMethod);
    }
    Class<?> clazz = testClass;
    while (testClass.getSuperclass() != null) {
      Method[] methods = clazz.getMethods();
      for (Method method : methods) {
        if (method.getName().equals(testMethod.getName())
            && method.isAnnotationPresent(DisplayName.class)) {
          return method.getAnnotation(DisplayName.class).value();
        }
      }
    }
    return super.generateDisplayNameForMethod(testClass, testMethod);
  }
}

In addition @QuarkusMainIntegrationTest could be extended to use this DisplayNameGenerator per default (but I'm not sure whether this should be a default behaviour):

@Target(ElementType.TYPE)
@ExtendWith({ QuarkusMainIntegrationTestExtension.class })
@DisplayNameGeneration(InheritedDisplayNameDisplayNameGenerator.class)
@Retention(RetentionPolicy.RUNTIME)
public @interface QuarkusMainIntegrationTest {
}

Btw: Perhaps worth considering: @DisplayNameGeneration is inherited

quarkusio / quarkus

Quarkus doen't use @DisplayName for Failsafe reports #14004