search

sbabcoc / TestNG-Foundation

TestNG Foundation is a lightweight collection of TestNG listeners, interfaces, and static utility classes that supplement and augment the functionality provided by the TestNG API.
Apache License 2.0
13 stars 5 forks source link

RETRY: Design a mechanism to enable inspection and reporting of failed test iterations #54

Closed sbabcoc closed 2 years ago

sbabcoc commented 2 years ago

The current implementation of RetryManager makes no attempt to report or preserve exceptions from failed invocations of retried tests. This loss of diagnostic information can obscure intermittent issues like communication timeouts and race conditions that could otherwise be identified and corrected. To avoid this loss of failure information, a mechanism should be developed to record exceptions from failed invocations of retried tests. A new interface will be implemented to enable implementers to access the list of recorded exceptions and to determine the final disposition of the failed method.

sbabcoc commented 2 years ago

There's actually nothing to do here. TestNG marks the result for the test being retried as "SKIPPED (was retried)", and the exception in in there, too. Registered test listeners get notified of the skipped invocation.