Tester does log a set of parameters for a test alongside their values for a given iteration, but it may not be enough to understand what's going on, since parameter names are not always self-documenting. We do (at least should) have parameter descriptions in the test source as Doxygen comments. We do already generate tests-info.xml by parsing the test sources, so the task is:
[ ] augment engine/builder/te_tests_info.sh to extract parameter descriptions
[ ] optionally validate that all test parameters actually used are documented and all @param names correspond to actual parameters
[ ] augment Tester to get those descriptions from tests-info.xml and log them
[ ] phase 2: also allow extraction and logging of enum values
It's all pretty straightforward but there is a slight challenge in dealing with env, since tests normally do not access env directly, but rather its components such as pco_iut, so there should be some heuristics to guess which test source parameters correspond to which package.xml parameters.
Tester does log a set of parameters for a test alongside their values for a given iteration, but it may not be enough to understand what's going on, since parameter names are not always self-documenting. We do (at least should) have parameter descriptions in the test source as Doxygen comments. We do already generate
tests-info.xml
by parsing the test sources, so the task is:engine/builder/te_tests_info.sh
to extract parameter descriptions@param
names correspond to actual parameterstests-info.xml
and log themIt's all pretty straightforward but there is a slight challenge in dealing with
env
, since tests normally do not accessenv
directly, but rather its components such aspco_iut
, so there should be some heuristics to guess which test source parameters correspond to whichpackage.xml
parameters.