ukri-excalibur / excalibur-tests

Performance benchmarks and regression tests for the ExCALIBUR project
https://ukri-excalibur.github.io/excalibur-tests/
Apache License 2.0
18 stars 15 forks source link

Make postprocessing work with default ReFrame perflogs #316

Open ilectra opened 5 months ago

ilectra commented 5 months ago

In general, the postprocessing doesn't depend on a specific format of the perflogs - at least that was the design plan. It does have some minimal requirements though, that should be inconsequential, but unfortunately they are not: they make it incompatible with the default perflogs produced by ReFrame (outside this framework) - as of version 4.5 at least. The reason is that among the few required columns is display_name, whereas the default perflogs contain short_name instead, which includes the same information.

It would be nice if our postprocessing would work seamlessly with default ReFrame perflogs, but it requires a bit more thinking than just changing the name of the required column, because ReFrame has a few variables to encode the benchmark name, and there is no guarantee which one(s) of those might stay in the default perflog in future versions.