How to display coverage number and trend from XML data

[godzillasaurus]

What would be the recommended tool/plug-in to be able to display current functional coverage numbers and coverage trend from the generated XML file in Jenkins? Does the XML data need to be parsed by another tool before it can be picked up by Jenkins plugins like JUnit or Cobertura? Please make me smart as I am not familiar with the post sim flow. Thank you in advance.

[mciepluc]

Hi @godzillasaurus I usually use an online converter to export XML into HTML format. Example sites: http://www.xmlviewer.org/ https://jsonformatter.org/xml-viewer Let me know if that works for you, or you find XML coverage database format not very useful and looking for something different (YAML?)

[themperek]

If one would like to use standard tools should somehow adopt Cobertura example: https://gist.github.com/apetro/fcfffb8c4cdab2c1061d and https://raw.githubusercontent.com/cobertura/web/master/htdocs/xml/coverage-04.dtd but I am not sure if this can apply to functional coverage.

For in python manipulation /combining/ regression maybe yaml would be better.

[majuscule]

Hi @mciepluc,

The goal here is usage of the coverage report with third party tooling. An example is the Jenkins cobertura-plugin. The Cobertura format (spec linked by @themperek) is also standardized in Python development, with tools such as pytest-cov generating coverage.xml utilizing the format.

This would also likely be a first step to resolve issue #18, as Azure pipelines also consume Cobertura formatted coverage reports (ref).

Thanks for looking into it!

[mciepluc]

I am not sure how to export functional coverage database to Cobertura format. I checked, and this format is typical for software code coverage presentation. If anyone has any ideas, please let me know.

But definitely we may easily add YAML functionality, which I understand may be helpful in many cases.

[themperek]

Maybe we use YAML as our functional proper coverage but can write some simple converter to Cobertura that would allow us to trace changes with standard tools (even if this will not be line numbers in our case)?

[godzillasaurus]

In addition to XML or YAML output, an option to directly generate report in html format would be welcome, similar to the functionality provided by xml_report() from Coverage.py

https://coverage.readthedocs.io/en/v4.5.x/api_coverage.html

[mciepluc]

Hi @godzillasaurus , As there are plenty of options of converting XML to HTML outside this framework, I do not see a need to have it in place right here. Maybe anyone has different suggestions regarding this, so it can be done very easily.

[cmarqu]

We could probably have accompanying XSL and CSS files to have the XML render nicely in a browser.

[cmarqu]

https://www.amiq.com/consulting/2018/11/15/new-release-of-the-functional-coverage-for-systemc-library/ can write UCIS, merge coverage databases, and has a pretty nice HTML GUI for cumulative coverage results. https://github.com/amiq-consulting/fc4sc/blob/master/doc/FC4SC_User_Guide.pdf shows the rendered result. License is Apache v2.

[mciepluc]

@themperek any success/plans with Cobertura?

[themperek]

@mciepluc Unfortunately not. It is a bit hacking I can try to work on this a bit but not in January. Maybe UCIS makes more sense. Especially if we can go with open tools.