microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
66.83k stars 3.66k forks source link

[Feature]: Update test.describe location in HTML reports #30698

Open loop-automation opened 6 months ago

loop-automation commented 6 months ago

🚀 Feature Request

Summary: Improve the readability and context provision in HTML reports by modifying the display behavior of test.describe.

Current Behavior: Currently, test.describe is appended before each test name in the file that uses test.describe. This implementation can become repetitive and somewhat limits the ability to provide extensive context due to character constraints.

Proposed Change: Update the test.describe display so that it appears within the grey box of the test file name as a paragraph (p). This adjustment aims to enhance the readability of the HTML report and, more importantly, allows for a significantly higher character limit. This change would enable users to provide more detailed context for each test file, aiding better understanding and maintenance of test suites.

Rationale: This enhancement would make the test.describe display less repetitive and more centralized, offering a cleaner and more informative layout in the HTML reports. Increasing the character limit for descriptions would allow for more comprehensive explanations, which is crucial for complex test suites or when tests are reviewed by multiple team members or stakeholders.

Expected Benefits:

Improved readability and aesthetics of HTML reports. Enhanced ability to provide detailed context for test files. Reduction in redundancy and clutter associated with multiple, repeated descriptions.

Screenshot 2024-05-07 at 3 03 08 PM

Similar behavior to test.info below:

Screenshot 2024-05-07 at 3 06 17 PM

Example

I would like to be able to use test.describe like below but currently the display limits the value of it.

test.describe("This file includes all tests for the Cyber security report section "Alerts", pages 6-9", () => { })

Motivation

Expected Benefits:

Improved readability and aesthetics of HTML reports. Enhanced ability to provide detailed context for test files. Reduction in redundancy and clutter associated with multiple, repeated descriptions.

GoldenGuts commented 6 months ago

What happens if there are multiple describe blocks in test file? The example you have given only accounts for one describe block. What I suggest is we can have multiple dropdowns for each describe block, applicable only if there are multiple blocks.

BMayhew commented 6 months ago

I wonder if rather than change the describe works, annotations that are created on the describe level could be exposed on the main report in some fashion. This was new functionality released with 1.42 https://playwright.dev/docs/test-annotations#annotate-tests

loop-automation commented 6 months ago

@GoldenGuts I think that is a reasonable solution. Some kind of display of the different grouping.

@BMayhew I actually didn't know about this and definitely feel like it is a step forward! I just wish this was displayed in some capacity under the fileName from the main view.

GoldenGuts commented 6 months ago

I was looking at the implementation on how it is displaying the test name. If there are nested blocks, it will show all the previous paths. [...test.path, test.title].join(' › ') In this, test.path is an array which has every text related to test. It would be very difficult to make a tree of all the nested describe blocks. If it's just one, then maybe we can create dropdown like we have on steps page. I can try to write code to ensure we have something meaningful, but before I commit, I'm open to suggestions on how we can make it better!

loop-automation commented 6 months ago

@GoldenGuts interesting. I wonder if the fact there can be multiple test.describes makes the feature not really applicable as the point of implementation. That being said, maybe a mechanism to add a description to the file that displays?