reframe-hpc / reframe

A powerful Python framework for writing and running portable regression tests and benchmarks for HPC systems.
https://reframe-hpc.readthedocs.org
BSD 3-Clause "New" or "Revised" License
224 stars 104 forks source link

[enhancement] Improve naming of automatically generated tests #3265

Closed vkarak closed 2 months ago

vkarak commented 2 months ago

The name test attribute is not anymore a full alias of display_name. It hides all special parameters starting with $ that are inserted by the automatic test generation options, such as --repeat, --distribute etc.

Also the test name of the parameterized test with --distribute does not contain any more the partition name, but instead the partition has become a special test parameter.

Finally, all the special parameters are converted to non-loggable.

The motivation behind these changes is that previously both the special parameters and the different name of the "distributed" tests were causing confusion with logging (e.g., different log files, different headers etc.) and were preventing results comparisons with the new --performance-compare option.