testng-team / testng

TestNG testing framework
https://testng.org
Apache License 2.0
1.99k stars 1.02k forks source link

Ability to overwrite test and suite summaries (to console) not available? #2228

Closed GandhiTC closed 4 years ago

GandhiTC commented 4 years ago

I just may not be aware of the info/knowledge, and if that is the case, I can close this issue. Can the default test and suite summaries to console not be overwritten or removed?

TestNG Version

Any 7.1.0 7.0 6.14.3

Expected behavior

Implementation of listener(s) to overwrite/format/remove/customize default test and/or suite summaries.

Actual behavior

Using a listener which implements ITestListener & ISuiteListener and overriding the onFinish() methods, I was only able to add custom test and suite summaries, but not remove or overwrite/replace the default ones. I've also looked into IReporter and logger, but it is my understanding that IReporter is more for the output files and not so much for the console output, and from what I could gather about logger, it can not do what I'm looking for.

Is the issue reproductible on runner?

Follow up

Thank you for your suggestion, but it did not help to hide/remove/alter the default summaries either. I ended up just implementing IAlterSuiteListener to forcefully set suite and test verbose levels to 0.

krmahadevan commented 4 years ago

@GandhiTC - Its not clear what your ask is.

The following are some of the default reporters that can be disabled.

To disable them you would configure your suite fire plugin to look like below

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>3.0.0-M1</version>
    <executions>
        <execution>
            <phase>test</phase>
        </execution>
    </executions>
    <configuration>
        <properties>
            <property>
                <name>usedefaultlisteners</name>
                <value>false</value>
            </property>
        </properties>
    </configuration>
</plugin>

Now to customize your reports the way you want, you can always build your own implementation that starts off by implementing either the

and then wiring them using

Here's a sample

import java.util.List;
import java.util.Set;
import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;
import org.testng.Reporter;

public class ConsoleLoggingListener implements ITestListener {

  @Override
  public void onFinish(ITestContext context) {
    System.err.println("Printing test reports for test <" + context.getName() + ">");
    report(context.getPassedTests().getAllResults(), "passed");
    report(context.getFailedTests().getAllResults(), "failed");
    report(context.getSkippedTests().getAllResults(), "skipped");
  }

  private void report(Set<ITestResult> results, String status) {
    if (results.isEmpty()) {
      return;
    }
    System.err.println("Test methods that " + status + " are as below");
    results.forEach(this::report);
  }

  private void report(ITestResult result) {
    String name = result.getMethod().getQualifiedName() + "()";
    System.err.println(name);
    List<String> output = Reporter.getOutput(result);
    if (!output.isEmpty()) {
      System.err.println("Test Method logs ");
      output.forEach(System.err::println);
    }
    if (result.getStatus() == ITestResult.SUCCESS) {
      return;
    }
    if (result.getStatus() == ITestResult.FAILURE) {
      System.err.println("Test Method failed due to ");
      result.getThrowable().printStackTrace();
    }
    if (result.getStatus() == ITestResult.SKIP) {
      System.err.println("Test Method skipped due to problems in the following methods");
      result
          .getSkipCausedBy()
          .forEach(iTestNGMethod -> System.err.println(iTestNGMethod.getQualifiedName()));
    }
  }
}

Closing this issue with resolution as Question Answered

If its not what you are looking for, please add more context to your question and we can reopen this on need basis.