allure-framework / allure-python

Allure integrations for Python test frameworks
https://allurereport.org/
Apache License 2.0
713 stars 233 forks source link

Allow users to actually take advantage of passed categories via message regex. #790

Open dsayling opened 5 months ago

dsayling commented 5 months ago

[//]: # ( . Note: for support questions, please use Stackoverflow or Gitter. . This repository's issues are reserved for feature requests and bug reports. . . In case of any problems with Allure Jenkins plugin please use the following repository . to create an issue: https://github.com/jenkinsci/allure-plugin/issues . . Make sure you have a clear name for your issue. The name should start with a capital . letter and no dot is required in the end of the sentence. An example of good issue names: . . - The report is broken in IE11 . - Add an ability to disable default plugins . - Support emoji in test descriptions )

As a user of the categories.json, I want to be able to display "Passed" categories mapped from the categories.json using the messageRegex, so that I can show different "Passed" categories by Name in the UI.

I'm submitting a ...

What is the current behavior?

In pytest_runtest_makereport, the default value of status_details is None, and when test_result.status == Status.PASSED, it's still None.

As far as I know there's no way to actually provide a message that would enable this to work in a categories.json

[
  {
    "name": "Integration Passed",
    "messageRegex": ".*IntegrationTest.*",
    "matchedStatuses": ["passed"]
  }
]

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

A simple bit of code I would expect to work

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
    report = (yield).get_result()
    if call.when == "call" and report.passed:
        report.longrepr = "IntegrationTest Passed"

What is the expected behavior?

That I can provide a Passed Message to allure to do stuff with the categories.json

        status_details = StatusDetails(message=report.longreprtext)

Please tell us about your environment:

platform darwin -- Python 3.11.3, pytest-7.4.2, pluggy-1.3.0
plugins: cov-4.1.0, mock-3.11.1, anyio-3.7.1, xdist-3.3.1, check-2.2.2, allure-pytest-2.13.2

Other information

None

[//]: # ( . e.g. detailed explanation, stacktraces, related issues, suggestions . how to fix, links for us to have more context, eg. Stackoverflow, Gitter etc )