allure-framework / allure-python

Allure integrations for Python test frameworks
https://allurereport.org/
Apache License 2.0
713 stars 233 forks source link

Suites view - failed Playwright pytest tests are not shown under their corresponding suite/sub-suite #788

Closed harmin-parra closed 6 months ago

harmin-parra commented 6 months ago

[//]: # ( . Note: for support questions, please use Stackoverflow or Gitter. . This repository's issues are reserved for feature requests and bug reports. . . In case of any problems with Allure Jenkins plugin please use the following repository . to create an issue: https://github.com/jenkinsci/allure-plugin/issues . . Make sure you have a clear name for your issue. The name should start with a capital . letter and no dot is required in the end of the sentence. An example of good issue names: . . - The report is broken in IE11 . - Add an ability to disable default plugins . - Support emoji in test descriptions )

I'm submitting a ...

The test:

  1. Install these python allure plugins:
    • allure-pytest
    • allure-behave
  2. Implement passing and failing test cases:
    • Behave tests
    • Regular pytest unit tests (without Playwright)
    • Playwright pytest unit tests
  3. Execute the tests and generate the report
  4. Go to the Suites view Screenshot 2024-01-14 at 19 51 46

What is the current behavior?

The failed regular pytest unit test and behave test are shown under their corresponding suite/sub-suite.

This is not the case for the failed Playwright pytest unit test, which is shown under a suite called Failure test instead of being shown under its corresponding suite/sub-suite (Wikipedia website in the example shown in screenshot).

The behavior is not coherent.

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

import allure
from playwright.sync_api import Page

@allure.parent_suite("GUI Tests")
@allure.suite("Wikipedia website")
def test_chinese(page: Page):
    with allure.step("Step 1"):
        pass  # do something
    with allure.step("Step 2"):
        pass  # do something
    assert False

What is the expected behavior?

The failed Playwright test should be displayed under its corresponding suite/sub-suite (Wikipedia website in the example shown in screenshot).

What is the motivation / use case for changing the behavior?

Have a coherent behavior across the different Python allure plugins.

If I wanted to have direct access to failed tests, I can always go to the Categories view.

Why regrouping the failed Playwright pytest tests separately in the Suites view?

Please tell us about your environment:

Other information

[//]: # ( . e.g. detailed explanation, stacktraces, related issues, suggestions . how to fix, links for us to have more context, eg. Stackoverflow, Gitter etc )

harmin-parra commented 6 months ago

I made a mistake in my test. Everything works fine.