search

mciepluc / cocotb-coverage

Functional Coverage and Constrained Randomization Extensions for Cocotb
BSD 2-Clause "Simplified" License
100 stars 15 forks source link

Support for UCIS coverage format #29

Open cmarqu opened 4 years ago

cmarqu commented 4 years ago

https://www.amiq.com/consulting/2018/11/15/new-release-of-the-functional-coverage-for-systemc-library/ can write UCIS, merge coverage databases, and has a pretty nice HTML GUI for cumulative coverage results. https://github.com/amiq-consulting/fc4sc/blob/master/doc/FC4SC_User_Guide.pdf shows the rendered result. License is Apache v2.

Originally posted by @cmarqu in https://github.com/mciepluc/cocotb-coverage/issues/22#issuecomment-563161873

cmarqu commented 4 years ago

I have started working on an export_to_ucis() method.

mciepluc commented 4 years ago

Cool @cmarqu , thanks! Do you know if anyone from Big Three is seriously working using with UCIS?

cmarqu commented 4 years ago

Do you know if anyone from Big Three is seriously working using with UCIS?

I don't. However, AMIQ are saying in https://www.amiq.com/consulting/2018/02/22/cpp-implementation-of-functional-coverage-for-systemc/

The FC4SC saves collected data in UCIS format in order to be compatible with functional coverage tools provided by 3rd party vendors (e.g. Cadence, Mentor, Synopsys).

cmarqu commented 4 years ago

@mciepluc UCIS has range as a first class citizen for a coveragepoint bin (see e.g. here: https://github.com/amiq-consulting/fc4sc/blob/35b80717092abef8f59af2bbd2f78d162a598ff5/examples/fir/coverage_results_gold.xml#L61-L63). cocotb-coverage handles ranges a bit indirectly with rel (as I read in your DVCon paper). This means that I cannot easily get the bounds of the ranges thus implemented from a CoverItem, right? Would it be useful to add some convenience code so that range bounds can be directly given (and queried)?

mciepluc commented 4 years ago

@cmarqu I think you are correct. Unfortunately UCIS seems to support only value/range bins, while cootb-coverage can accept any relation scheme. Bins matching is according to the rule: relation(var, bins) = True. The problem is that bin/variable does not have to be an integer... any ideas?

cmarqu commented 4 years ago

I suppose they only added to UCIS what could be done with the languages and tools at that time (it was primarily meant to exchange coverage data between different vendors' tools I think), so we may be out of luck. There are userAttr subelements for some XML elements, maybe we can use these. I'm also not sure what exactly the exprString could do for us, right now, I'm (mis)using it to display vname, rel and transformation.

mciepluc commented 4 years ago

@cmarqu for me userAttr looks ok to include all things we need, both in coverpoint and coverbin. It can contain multiple items. I would not use the exprString it seems to be dedicated for something else.

cmarqu commented 4 years ago

It looks like I need to get more familiar with the CoverItems etc. themselves before I can make progress on this, so I'll first port over (parts of) your paper to Sphinx to learn as I go.

mciepluc commented 4 years ago

@cmarqu do you have any work done to share? I may have some time to work on this next weeks.

cmarqu commented 4 years ago

@mciepluc It's in https://github.com/mciepluc/cocotb-coverage/compare/master...cmarqu:add-ucis-support. There is no automated checks for UCIS, I was using the AMIQ fc4sc HTML renderer to look at the UCIS output.