teemtee / tmt

Test Management Tool
MIT License
79 stars 117 forks source link

Implement linting for plan steps #884

Open FrNecas opened 2 years ago

FrNecas commented 2 years ago

Currently, only top-level (in terms of fmf hierarchy) attributes in plans are linted. For example the linter throws an unknown key error on the following plan:

prvision:
    how: container
execute:
    how: tmt
    script: tmt --help

but the following plan is OK according to the linter:

provision:
    how: container
    imge: fedora
execute:
    how: tmt
    script: tmt --help
idorax commented 1 year ago

Looks this issue is fixed in tmt-1.21.0, e.g.

$ cat plans/foo884.fmf
prvision:
    how: container
execute:
    how: tmt
    script: tmt --help
$ tmt plans lint
    warn: /plans/foo884: - Additional properties are not allowed ('prvision' was unexpected)
/plans/foo884
fail unknown attribute 'prvision' is used
warn summary is very useful for quick inspection

Hi @psss and @lukaszachy , can we close this issue?

lukaszachy commented 1 year ago

I don't think so, see the second example:

provision:
    how: container
    imge: fedora
execute:
    how: tmt
    script: tmt --help

The tmt plans lint exits with 0 (== all good). All what shows that something is wrong is a warning, but then it says 'pass correct attributes are used':

warn: /p:provision - {'how': 'container', 'imge': 'fedora'} is not valid under any of the given schemas
/p
pass correct attributes are used
warn summary is very useful for quick inspection
lukaszachy commented 1 year ago

Note that there is ongoing plan to utilize schema validation for this (which indeed reports the warning) however tmt allows user to safe a lot of typing by allowing unique prefix for plugin values (the how: value) which doesn't work well for the validation (#1447)