[//]: # (
. Note: for support questions, please use Stackoverflow or Gitter.
. This repository's issues are reserved for feature requests and bug reports.
.
. In case of any problems with Allure Jenkins plugin please use the following repository
. to create an issue: https://github.com/jenkinsci/allure-plugin/issues
.
. Make sure you have a clear name for your issue. The name should start with a capital
. letter and no dot is required in the end of the sentence. An example of good issue names:
.
. - The report is broken in IE11
. - Add an ability to disable default plugins
. - Support emoji in test descriptions
)
I'm submitting a ...
[x] bug report
[ ] feature request
[ ] support request => Please do not submit support request here, see note at the top of this template.
What is the current behavior?
Tests that are re-run with pytset-rerunfailures are reported as multiple failures, while the "Retries" tab remains empty - as visible on the attached screenshot.
If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem
The contents of the test script, in this case test_me_2.py:
def test_fail():
assert False
Execute pytest --reruns=2 --alluredir=<allure_dir> test_me_2.py
And then display the allure report.
Tests are reported once, in this case it is obviously a failure by design, the re-runs end up in the "Retries" section as on the screenshot below. This is the report I'm seeing when using allure-pytest-2.12.0:
What is the motivation / use case for changing the behavior?
The plugin pytest-rerunfailures is a common pytest plugin to make an attempt at re-running flaky tests. Allure used to work perfectly with it, now it doesn't.
Please tell us about your environment:
Allure version: 2.10.0
Test framework: pytest@7.2.2
Allure adaptor: allure-pytest@2.13.0
Other information
[//]: # (
. e.g. detailed explanation, stacktraces, related issues, suggestions
. how to fix, links for us to have more context, eg. Stackoverflow, Gitter etc
)
[//]: # ( . Note: for support questions, please use Stackoverflow or Gitter. . This repository's issues are reserved for feature requests and bug reports. . . In case of any problems with Allure Jenkins plugin please use the following repository . to create an issue: https://github.com/jenkinsci/allure-plugin/issues . . Make sure you have a clear name for your issue. The name should start with a capital . letter and no dot is required in the end of the sentence. An example of good issue names: . . - The report is broken in IE11 . - Add an ability to disable default plugins . - Support emoji in test descriptions )
I'm submitting a ...
What is the current behavior?
Tests that are re-run with pytset-rerunfailures are reported as multiple failures, while the "Retries" tab remains empty - as visible on the attached screenshot.
If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem
The contents of the test script, in this case
test_me_2.py
:Execute
pytest --reruns=2 --alluredir=<allure_dir> test_me_2.py
And then display the allure report.The environment (venv) contents:
What is the expected behavior?
Tests are reported once, in this case it is obviously a failure by design, the re-runs end up in the "Retries" section as on the screenshot below. This is the report I'm seeing when using allure-pytest-2.12.0:
What is the motivation / use case for changing the behavior?
The plugin pytest-rerunfailures is a common pytest plugin to make an attempt at re-running flaky tests. Allure used to work perfectly with it, now it doesn't.
Please tell us about your environment:
Other information
[//]: # ( . e.g. detailed explanation, stacktraces, related issues, suggestions . how to fix, links for us to have more context, eg. Stackoverflow, Gitter etc )