It should be possible to have description of session/run

oktetlabs / test-environment

OKTET Labs Test Environment

Other

5 stars 9 forks source link

It should be possible to have description of session/run #50

Open okt-kostik opened 2 weeks ago

okt-kostik commented 2 weeks ago

Description is very useful and looks super nice in bublik

thanks for adding it. Said that we can't specify it for runs and session. Thus it leads to empty descriptions (see above).

I think it should be trivial to support description all over the place. Please help

ol-andrewr commented 2 weeks ago

It is called "objective" in the test and I think we should preserve terminology here.

"run" and "session" exist in packange.xml only and it means that objective should be added there.

If run specifies an objective, it should override the test script objective. Or do we want more complicated handling with possibility to extend test script objective?

okt-kostik commented 2 weeks ago

Let's call it objective. I think description is the name used in the package.

I think we should keep it simple:

run with objective overrides script

ol-dmitryiz commented 1 week ago

It appears we already have <description> and <objective> tags in package.xml. <description> can be used in <package>, however contents is saved in pkg->objective. <objective> can be used in <script> and is parsed into script->objective. In log.xml they both go to <objective> tag.

However if I try to set <script>/<objective> in package.xml to something different than what is written under @objective in test C file, complaint is printed in log: “Inconsistency in test 'ip4_raw_send' objective from package.xml and tests-info.xml”. And still content from @objective is used. This is easy to fix in Tester.

Only for <session> we have neither <description> nor <objective>.

okt-kostik commented 1 week ago

We do have description and objective, but they can't be specified for run and session, right?

Only package and script.

ol-dmitryiz commented 1 week ago

For <run> we do not have it and I doubt we should. It seems enough to add <objective> (or <description>) for <session> - the only remaining thing which can be found inside <run> and does not have such tag yet.

ol-andrewr commented 1 week ago

Unfortunately not. Sometimes different runs are used for the same script and it is desirable to highlight particular run specific objective. It should override script/session objective.

ol-dmitryiz commented 1 week ago

You will be able to specify different objective in every <script> inside <run>. These will be different <script> tags inside different <run> tags.

ol-andrewr commented 1 week ago