str0zzapreti / pytest-retry

A simple plugin for retrying flaky tests in CI environments
MIT License
27 stars 6 forks source link

feature: improve simple test reporting #5

Closed str0zzapreti closed 1 year ago

str0zzapreti commented 1 year ago

Overhauls the internal reporting system to make more detailed information about the test state available during and after the runtest protocol. Initially, items only stored a simple True/False value for success key and a count for the total number of attempts. Now, items store a str outcome which can be passed, failed, or skipped. A new duration key has also been added which enables easy access to test duration from hooks (note: duration is only calculated after item teardown has been completed and is not accessible sooner).

Possible future changes: a rolling sum for duration to enable partial reporting (e.g. make duration available when only setup or setup and call have completed)

Added some new tests to cover the additional changes