Allow appending `exclude` fields in discover phase from CLI

mkoncek commented 1 year ago

Given a test plan whose discover phase already contains exclusions I would like to be able to append additional ones when invoking tmt. The syntax can look like: tmt ... discover --exclude+ <FIELD> ....

psss commented 1 year ago

Thanks for the idea! Sounds like a nice and useful improvement.

The syntax can look like: tmt ... discover --exclude+ <FIELD> ....

Using + for extending exclude would be consistent with how it's done in fmf, however it seems the special character is not supported by click. So maybe one of these?

tmt run ... discover --exclude-add
tmt run ... discover --exclude-append
tmt run ... discover --exclude-extend

Should be fairly easy to implement.

mkoncek commented 1 year ago

If exclusions are internally stored as a set then --exclude-add, if they are stored as a list, then probably a duo of --exclude-prepend, --exclude-append.

idorax commented 1 year ago

Hi @mkoncek and @psss, I'd like to confirm the expected output if we run tmt ... discover --exclude-add ....

bash$ cat plans.fmf 
execute:
    how: tmt

/smoke:
    provision:
        how: local

/fmf:
    /exclude:
        discover:
            how: fmf
            exclude: /tests/discover1
bash$ cat tests.fmf 
test: /bin/true

/discover1:
    summary: Discover one
    tier: 1
    link: /tmp/foo

/discover2:
    summary: Discover two
    tier: 2
    link:
    - note: Some text
      verifies: https://github.com/teemtee/tmt/issues/870

/discover3:
    summary: Discover three
    tier: 3
bash$ tmt run discover --how fmf plans --name 'fmf/exclude' -v
/var/tmp/tmt/run-065

/data/plans/fmf/exclude
    discover
        how: fmf
        directory: /home/huanli/dev/1465/foo
        summary: 2 tests selected
            /data/tests/discover2
            /data/tests/discover3
bash$ tmt run discover --how fmf --exclude discover2 plans --name 'fmf/exclude' -v
/var/tmp/tmt/run-066

/data/plans/fmf/exclude
    discover
        how: fmf
        directory: /home/huanli/dev/1465/foo
        summary: 2 tests selected
            /data/tests/discover1
            /data/tests/discover3
# ^^^ /data/tests/discover1 is not excluded !!!

If we run tmt run discover --how fmf --exclude-add discover2 plans --name 'fmf/exclude' -v, the output looks like:

/data/plans/fmf/exclude
    discover
        how: fmf
        directory: /home/huanli/dev/1465/foo
        summary: 1 test selected
            /data/tests/discover3
# ^^^ both  /data/tests/discover1
# ^^^  and  /data/tests/discover2 
# ^^^  are excluded !!!

Is the output above what you exactly expect to have ?

teemtee / tmt

Allow appending `exclude` fields in discover phase from CLI #1465