allure-framework / allure-python

Allure integrations for Python test frameworks
https://allurereport.org/
Apache License 2.0
719 stars 235 forks source link

Add labels from env for all tests in a run #721

Open delatrie opened 1 year ago

delatrie commented 1 year ago

I'm submitting a ...

What is the current behavior?

Currently there is no way to pass environment-related info into all tests in a run without modifying the code base of the tests.

What is the expected behavior?

Allure should automatically convert environment variables to labels using the naming convention specified below.

What is the motivation / use case for changing the behavior?

It`s useful to test an application or library across multiple different environments. Such environments may differ in

The environment-based labels help to distinguish test results produced by the same test in different environments. TestOps can group those results apart with its env from label feature.

Other information

Naming convention

Every environment variable in form ALLURE_LABEL_<KEY>=<value> is translated into the label <KEY> with the value <value>. The label is added for each test in the run.

Constraints

To ensure consistency across different allure adapters the following constraints are imposed:

Example

Given the following environment variables:

ALLURE_LABEL_PYTHON_VERSION=3.11
ALLURE_LABEL_OS=Linux
ALLURE_LABEL_USERNAME=John
ALLURE_LABEL_EMPTY=

When the tests are run.

Then the following labels are present in any *-result.json file created during the run:

{
  ...
  "labels": [
    ...
    {
      "name": "PYTHON_VERSION",
      "value": "3.11"
    },
    {
      "name": "OS",
      "value": "Linux"
    },
    {
      "name": "USERNAME",
      "value": "John"
    },
    {
      "name": "EMPTY",
      "value": ""
    }
  ]
}
cveal-Momnt commented 4 months ago

Hi! This feature is very needed for my team. It would allow us to work around the allure testops filtering limitations without needing to modify codebase.

Is this being worked on?