teemtee / tmt

Test Management Tool
MIT License
82 stars 123 forks source link

RFE: name format - ability not to use regexp for tests selection/discovery #752

Open sopos opened 3 years ago

sopos commented 3 years ago

The help message says it is simply a REGEXP.

The actual behavior is: --name / - matches everything --name . - matches everything under the current path

But actually . matches everything starting with the current path otherwise . would basically have to match anything.

It would be more understandable not to use regexp by default but a real path reference where all the nodes underneath would be taken into a consideration. Thus e.g. /fapolicyd/Sanity/integrity and /fapolicyd/Sanity/integrity-advanced would be handled separately, not the former matching the latter as well as it is now.

Working with the directory structure, possibly with a globe pattern support, should cover most of the usecases.

A new option --regexp could be introduced to return the current behavior.

sopos commented 2 years ago

actually, it would be nice to be able to disable the regexp and flip it to the path/globbing approarch. That could be done in both command line and discover in a test plan.

tmt run tests --name '...' --match glob
tmt run tests --name '...' --match regexp; # <<<< the default
discover:
    name: xyz
    match: glob
psss commented 2 years ago

The new option should be usable with tmt test ls or tmt test show as well. Here the --format wording sounds a bit confusing as it could suggest the format of the listing/showing could be specified. Perhaps something like --search=glob and --search=regexp would be a bit more selfexplanatory?

As far as the discover step is concerned, tests are selected using the following way:

discover:
    how: fmf
    test: /tests/basic
    search: glob

Also --match could be a good name: Match test names using a regular expression or a file pattern globbing.

sopos commented 2 years ago

I like --search as well as --match. I'm leaning more towards --match as we are actually not searching we are selecting / matching against the pattern. I updated the example.