search

teemtee / tmt

Test Management Tool
MIT License
79 stars 117 forks source link

plans imports do not propagate context into imported plans #3035

Open pemensik opened 4 weeks ago

pemensik commented 4 weeks ago

I am trying to reuse both plans and tests for multiple components, which contain bind9 in various versions. Therefore common tests should be possible. I made it possible with use of context: component: bindX, as demonstrated in https://gitlab.com/redhat/centos-stream/rpms/bind9.18/-/merge_requests/3.

But either it is not sufficiently documented or it cannot propagate context changed from original place, which imports plans from different repository, which imports tests from a repository too.

End tests are using adjusts to require correct dependencies, based on context defined. Example of it is in https://src.fedoraproject.org/tests/bind/pull-request/13#.

But I have not found, how can I define context: component: in a fmf file containing plan: import:, which would also show in tmt plans show. If I use tmt -c component=bind9-next, it shows it fine. But not from a file containing import (or above hierarchy). Strange is environment is propagated fine, but component is not.

environment+:
  PACKAGE: bind9.18
context:
  component: bind9.18

/tier1-public-i:
  plan:
    import:
      url: https://src.fedoraproject.org/tests/bind.git
      name: /plans/tier1/public

/tier1-public:
  summary: Public (Fedora) Tier1 beakerlib tests
  discover:
    how: fmf
    url: https://src.fedoraproject.org/tests/bind.git
    filter: 'tier: 1'
  execute:
    how: tmt

Gives significant difference in results, imported has only environment propagated. I think adjust cannot be done based on that, but can be on component. Which does not seem to be propagated.

My request is to make propagation possible either implicitly or explicitly by listing propagated context items.

LecrisUT commented 3 weeks ago

Closest issue related to this would be: https://github.com/teemtee/tmt/issues/2964. Initially I was also considering having explicit support for each step, but I think that simply having something like adjust-plan would cover both cases. I think it's best to follow https://github.com/teemtee/tmt/pull/2865 which does something similar for adjust-test at which point there will be a clearer picture on how the plan.import could be designed.