serenity-bdd / serenity-core

Serenity BDD is a test automation library designed to make writing automated acceptance tests easier, and more fun.
http://serenity-bdd.info
Other
721 stars 517 forks source link

Why not have failed cases grouped by assertions? #3317

Open Rangiferus opened 1 year ago

Rangiferus commented 1 year ago

It would be very helpful in analysis of failed test cases to have grouping by assertion feature in Serenity reports. This functionality is provided by Allure Report and I've been using them extensively in my previous projects. In Allure it's highly customizable and the "Categories" applet is always present in reports. As they say:

Categories is one of the most time-saving features of Allure Report. It provides simple automation for fail resolution.

image

Apart from the default "product defects" (failed tests) and "test defects" (broken tests) a user can create own categories defined with messageRegex (e.g. ".element not clickable."), matchedStatuses (e.g. ["broken", "failed"]), traceRegex (e.g. ".FileNotFoundException.") Having such a feature you can quickly spot massive problems that are present in mulitple failed cases - not like currently in Serenity reports where you have to look at each failed case individually, expand it to the failed step to see the assertion message.

wakaleo commented 1 year ago

That's a good idea.