search

modelica-tools / csv-compare

Tool to compare curves from one csv files with curves from other csv files using an adjustable tolerance
https://www.modelica.org
BSD 3-Clause "New" or "Revised" License
26 stars 15 forks source link

Junit support #63

Open lionelbelmon opened 4 years ago

lionelbelmon commented 4 years ago

Since CSVcompare is typically used in CI toolchains for verifying non-regression of results, it would make sense to also generate a Junit XML summary of the comparisons.

beutlich commented 4 years ago

Can you please specify how such an XML summary is defined and where it needs to be saved? See https://stackoverflow.com/q/4922867/8520615.

Currently we have simple log files per comparison for automatic detection of the comparison verdict, see. e.g. https://github.com/modelica/MAP-LIB_ReferenceResults/blob/v4.0.0/Modelica/Blocks/Examples/PID_Controller/compare_passed.log for an example.

lionelbelmon commented 4 years ago

HI Thomas,

you get the right link.

the Junit is a summary report for all comparisons, so should be dumped at the root, near the log file.

you can test if the Junit XML works well with a Jenkins Junit plug-in.

the content could be 1 test suite where each CSV comparison is a test case.

the main benefit is that you get a standardized summary reports that can be parsed by other tools and used in dashboards.

Junit_format_xml.txt