allure-framework / allure-python

Allure integrations for Python test frameworks
https://allurereport.org/
Apache License 2.0
719 stars 235 forks source link

`allure.dynamic.title` does not create an appropriate title if fixture steps fail #730

Closed storenth closed 1 year ago

storenth commented 1 year ago

Describe the bug No appropriate title in case of test's fixture fail

To Reproduce

  1. Create failed fixture
  2. Add fixture to the test
  3. Add parametrized test with title param
  4. Add allure.dynamic.title based on parametrized title
  5. Run the test

Expected behavior Title of the failed test same as title in param

Screenshots Normally passed test:

allure-dynamic-test-pass

Test failed in test body (like assert False statement inside the test_case()):

allure-dynamic-assert-fail

Actual behavior Test failed in any fixture passed to the test_case(failed_fixture):

allure-dynamic-fixture-fail

Source code example including allure.dynamic.title with test params and fixtures:

allure-dynamic-source

Environment:

Allure version 2.20.1
Test framework pytest@7.2.1
Allure adaptor allure-pytest@2.12.0
delatrie commented 1 year ago

Hi, @storenth !

If a fixture fails, the test body is not executed by pytest, hence the allyre.dynamic.title function has no chance to change the title.

You can use @allure.title decorator instead, it has access to parameter values:

import allure
import pytest

@pytest.fixture
def failing_fixture():
    raise ValueError("Fixture failed")

@allure.title("{title}")
@pytest.mark.parametrize("title", ["First test", "Second test"])
def test_issue730_reproduction(failing_fixture, title):
    pass

image

storenth commented 1 year ago

Yes, it helps.

delatrie commented 1 year ago

You're welcome!