Currently, allure-pytest has a bunch of filters to run tests selectively:
--allure-epics
--allure-features
--allure-severities
--allure-stories
--allure-ids
We need to replace them with one general filtering mechanism based on labels.
Once this is done, the old filtering parameters could be deprecated in favor of the new one.
Motivation
Label-based test selection is more flexible in general. Since epics, features, stories, severities and ids are all labels, those filtering scenarios will be automatically supported.
In addition, other scenarios, like #633 will be supported automatically.
Additionally, custom labels (like ones, created from system env vars, see #721) would also be usable for test selection.
The syntax will become a little bit bulkier but it's always possible to wrap the invocation with a particular set of filters into a command runner like poetry or hatch.
Other information
New parameter may look like this:
--allure-label LABEL=VALUE
or
--allure-label-filter LABEL=VALUE (longer, but more expressive)
Conjunction of filters
It should be possible to specify the parameter multiple times to replace the following usage
where tests with either severity=blocker or severity=critical labels are selected.
Since all the current filters use the comma symbol to separate allowed values, we can maintain this behavior. Additionally, we should provide for possibility of a value itself containing a comma by implementing an escaping mechanism (currently this is not supported; but labels like feature, or story may potentially contain commas).
Example 1: selecting tests with severity either blocker or critical:
Currently,
allure-pytesthas a bunch of filters to run tests selectively:--allure-epics--allure-features--allure-severities--allure-stories--allure-idsWe need to replace them with one general filtering mechanism based on labels. Once this is done, the old filtering parameters could be deprecated in favor of the new one.
Motivation
Label-based test selection is more flexible in general. Since epics, features, stories, severities and ids are all labels, those filtering scenarios will be automatically supported. In addition, other scenarios, like #633 will be supported automatically. Additionally, custom labels (like ones, created from system env vars, see #721) would also be usable for test selection. The syntax will become a little bit bulkier but it's always possible to wrap the invocation with a particular set of filters into a command runner like
poetryorhatch.Other information
New parameter may look like this:
--allure-label LABEL=VALUEor--allure-label-filter LABEL=VALUE(longer, but more expressive)Conjunction of filters
It should be possible to specify the parameter multiple times to replace the following usage
with the new syntax:
This usage only selects tests matching both filters at the same time.
Disjunction of filters
It should be also possible to provide an alternative to calls like this:
where tests with either
severity=blockerorseverity=criticallabels are selected.Since all the current filters use the comma symbol to separate allowed values, we can maintain this behavior. Additionally, we should provide for possibility of a value itself containing a comma by implementing an escaping mechanism (currently this is not supported; but labels like feature, or story may potentially contain commas).
Example 1: selecting tests with severity either
blockerorcritical:Example 2: selecting tests for the features
Red, greenorPlus\minus: