search

teemtee / tmt

Test Management Tool
MIT License
79 stars 119 forks source link

[RFE] enable shorter test names in plans #1795

Open kkaarreell opened 1 year ago

kkaarreell commented 1 year ago

I had a test /setup/configure-repo-with-rpms-from-brew' defined in /setup/configure-repo-with-rpms-from-brew/main.fmf. I am using this test in my tmt plans.
Later, I wanted to introduced a parametrized version of this task that would be configuring swtpm repo. I have created /setup/configure-repo-with-rpms-from-brew/swtpm.fmf but from that moment /setup/configure-repo-with-rpms-from-brew stopped being a leaf object and therefore test /setup/configure-repo-with-rpms-from-brew does not exist. To restore original test I have defined /default/ in/setup/configure-repo-with-rpms-from-brew/main.fmf`, so I have tests:

/setup/configure-repo-with-rpms-from-brew/default
/setup/configure-repo-with-rpms-from-brew/swtpm

which I can use in my plans, however I need to update former place to use /setup/configure-repo-with-rpms-from-brew/default instead of /setup/configure-repo-with-rpms-from-brew.

It would be nice to be able to avoid such compatibility breakage and enable using /setup/configure-repo-with-rpms-from-brew in a tmt plan that would be automatically understood as /setup/configure-repo-with-rpms-from-brew/default if /setup/configure-repo-with-rpms-from-brew is not a leaf object.

Just idea, not pushing. :-) I did my plan updates already.

kkaarreell commented 1 year ago

I have just realized that this would break test name matching since

test:
 - /setup/configure-repo-with-rpms-from-brew

would in fact match both new tests while originally we wanted to run only one of them. In reality, we don't know what the test author wanted. :-(

We could enforce matching /setup/configure-repo-with-rpms-from-brew to /setup/configure-repo-with-rpms-from-brew/default only while /setup/configure-repo-with-rpms-from-brew/ would match with both but it would be not intuitive. Such enforcement could be defined on a test level so that consumers won't have to bother (e.g. they won't know if a new subtest have been created and most likely they do not want to run it).

Definitely needs more thinking...

lukaszachy commented 5 months ago

Definitely needs more thinking...

One can use safer way of picking test names, aka adding '$' to match whole suffix.

kkaarreell commented 5 months ago

Definitely needs more thinking...

One can use safer way of picking test names, aka adding '$' to match whole suffix.

Right. I was also thinking about that option but it would mean to update all relevant tes plans which I wanted to avoid. Seems like not having https://github.com/teemtee/tmt/issues/2002 hits us again.