search

teemtee / tmt

Test Management Tool
MIT License
82 stars 123 forks source link

Support easy context adjust for RHEL and all its derivatives #472

Open pcahyna opened 3 years ago

pcahyna commented 3 years ago

https://fmf.readthedocs.io/en/latest/context.html uses fedora and centos in its examples for relevancy filtering by distribution. I suppose rhel is also among the allowed values. This fine-grained filtering is not so useful in practice, because CentOS should be essentially identical to RHEL and tests that can run on one but not on the other should be an exception, not the norm. One should therefore have an easy way to express that a test can run on a given version of RHEL and CentOS, without requiring the test writers to constantly write distro >= centos-8 or distro >= rhel-8 or the like. I propose to inroduce a el alias that would match both CentOS and RHEL (and possibly any other RHEL derivative that uses the same numbering scheme, like Scientific Linux). Prior art is Ansible Galaxy, which uses a distro name of EL to mean RHEL and CentOS and probably others: see https://galaxy.ansible.com/docs/contributing/creating_role.html#role-metadata and note that RHEL and CentOS are even not in the list of supported platforms at https://galaxy.ansible.com/api/v1/platforms/ , but EL is. Since I mention Ansible, I want to clarify that I am referencing the prior art in the platform field of role metadata for Galaxy and not any prior art among the facts that Ansible collects, because there is nothing suitable. While ansible_os_family seems superficially similar, the problem with it is that it is set to RedHat even on Fedora, despite the completely different numbering scheme. I.e. using this field, distro >= RedHat-8 would match not only RHEL, CentOS 8 and above, but also all Fedora versions starting with Fedora Core 8 - definitely not what we want. In other words, the basic requirement for a distribution to be covered by the el alias is not only that it is sufficiently similar to RHEL, but that in addition the version numbers are compatible.

lukaszachy commented 3 years ago

I like this idea. I would say we can achieve this by setting 'el-X' values into distro dimension. If one wants "RHEL or CentOS" they will use distro == el. If one wants specific one they will use e.g. distro == rhel

As for the implementation: a) let TMT automatically add 'el-x' value to distro dimension every time centos, rhel is set there. or b) hardcode this logic to fmf.Context init implementation

Personally: option 'a)' but it has to work for autodetection and command line option as well (user should not remember to type it manually). E.g. --context 'distro=rhel-8.3.0' well be enough for tmt to create Context("distro=rhel-8.3.0,el-8.3.0")

psss commented 3 years ago

Make sense to support el as a general form. Agree with @lukaszachy, that the value should be auto-filled when specific distro is defined either on command line or within a plan config. This should be implemented in tmt rather than fmf which does not care about dimension names at all.

psss commented 3 years ago

@lukaszachy discovered an important regression in the approach used in #567:

$ cat main.fmf 
test: echo
adjust:
- when: distro != rhel-8
  enabled: false

$ No elX support:
$ tmt -c distro=rhel-8.4.0 tests show .
/
        test echo
        path /
      manual no
    duration 5m
     enabled yes
      result respect

# with elX support
$ tmt -c distro=rhel-8.4.0 tests show .
/
        test echo
        path /
      manual no
    duration 5m
     enabled no
      result respect

We need to revert and look into this more closely. @lukaszachy, good catch!