Allow code coverage collection

[Paebbels]

For many verification projects, the collection of (code) coverage information is essential.
Such a feature would complement OSVVM's latest feature on exporting and visualizing function coverage.

Commercials simulators like Riviera-PRO allow the collection of various coverage metrics like:

• statement coverage
• branch coverage
• expression coverage
• FSM coverage
• ...

OSVVM's scripting should offer a generic approach to enable code coverage collection. Collected coverage might need a "merge" step after testcase execution.

A basic example for Riviera-PRO can be offered upon request.

[Paebbels]

Compiling source files with coverage collection

ModelSim

+cover= enables coverage collection
AnalyzeCovTypes is a sequence of character for each coverage type like statement, branch, expression, ...

The changes for VHDL and Verilog are identical.

proc vendor_analyze_vhdl {LibraryName FileName OptionalCommands} {
  variable VhdlVersion
-  puts "vcom -${VhdlVersion} -work ${LibraryName} ${FileName} "
-        vcom -${VhdlVersion} -work ${LibraryName} ${FileName}
+  global AnalyzeCov
+  global AnalyzeCovTypes

+  if {$AnalyzeCov == "On"} then {
+    # with coverage options
+    puts "vcom -${VhdlVersion} +cover=${AnalyzeCovTypes} -work ${LibraryName} ${FileName} "
+          vcom -${VhdlVersion} +cover=${AnalyzeCovTypes} -work ${LibraryName} ${FileName}
+  } else {
+    # without coverage options (OSVVM defaults)
+    puts "vcom -${VhdlVersion} -work ${LibraryName} ${FileName} "
+          vcom -${VhdlVersion} -work ${LibraryName} ${FileName}
+  };
}

Riviera-PRO

-coverage enables coverage collection
AnalyzeCovTypes is a sequence of character for each coverage type like statement, branch, expression, ...

Coverage characters:

• s: statement
• b: branch
• f: functional
• e: expression
• c: condition
• t: toggle
• a: assertion
• p: path
• m: FSM

Example:
set AnalyzeCovTypes "sbecapm"

The changes for VHDL and Verilog are identical.

proc vendor_analyze_vhdl {LibraryName FileName OptionalCommands} {
  variable VhdlVersion
-  echo vcom -${VhdlVersion} -dbg -relax -work ${LibraryName} ${FileName}
-       vcom -${VhdlVersion} -dbg -relax -work ${LibraryName} ${FileName}
+  global AnalyzeCov
+  global AnalyzeCovTypes
+
+  if {$AnalyzeCov == "On"} then { 
+    # with coverage options
+    echo vcom -${VhdlVersion} -dbg -relax -coverage ${AnalyzeCovTypes} -work ${LibraryName} ${FileName}
+         vcom -${VhdlVersion} -dbg -relax -coverage ${AnalyzeCovTypes} -work ${LibraryName} ${FileName}
+  } else {
+    # without coverage options (OSVVM defaults)
+    echo vcom -${VhdlVersion} -dbg -relax -work ${LibraryName} ${FileName}
+         vcom -${VhdlVersion} -dbg -relax -work ${LibraryName} ${FileName}
+  };
}

Run simulation with coverage collection

A common -coverage switch is needed to enable a run with code coverage collection. Afterwards a coverage save is needed.

ModelSim

-  puts "vsim -voptargs='+acc' -t $SIMULATE_TIME_UNITS -lib ${LibraryName} ${LibraryUnit} ${OptionalCommands} -suppress 8683 -suppress 8684"
-#  eval vsim -voptargs="+acc" -t $SIMULATE_TIME_UNITS -lib ${LibraryName} ${LibraryUnit} ${OptionalCommands} -suppress 8683 -suppress 8684 
-  eval vsim -voptargs="+acc" -t $SIMULATE_TIME_UNITS -lib ${LibraryName} ${LibraryUnit} ${OptionalCommands} -suppress 8683 -suppress 8684
+  global SimCov
+  global SimCovTypes
+
+  if {$SimCov == "On"} then {
+    # with coverage options
+    puts "vsim -voptargs='+acc' -coverage -t $SIMULATE_TIME_UNITS -lib ${LibraryName} ${LibraryUnit} ${OptionalCommands} -suppress 8683 -suppress 8684"
+    eval vsim -voptargs="+acc" -coverage -t $SIMULATE_TIME_UNITS -lib ${LibraryName} ${LibraryUnit} ${OptionalCommands} -suppress 8683 -suppress 8684
+
+    # save coverage database
+    coverage save -onexit -code ${SimCovTypes} ${LibraryUnit}.ucdb
+  } else {
+    # without coverage options (OSVVM defaults)
+    puts "vsim -voptargs='+acc' -t $SIMULATE_TIME_UNITS -lib ${LibraryName} ${LibraryUnit} ${OptionalCommands} -suppress 8683 -suppress 8684"
+     eval vsim -voptargs="+acc" -t $SIMULATE_TIME_UNITS -lib ${LibraryName} ${LibraryUnit} ${OptionalCommands} -suppress 8683 -suppress 8684
+  };

Riviera-PRO

Riviera-PRO needs:

• simulation coverage options
• the coverage type sequence of characters
• a list of design units
  (Is there an option to collect for all of them?)

Setting coverage characters for simulation:
set SimCovTypes "sbfectapm"

Needed simulation coverage options are:
set SimCovOptions "-acdb -acdb_cov";

-  puts "vsim -t $SIMULATE_TIME_UNITS -lib ${LibraryName} ${LibraryUnit} ${OptionalCommands}"
-  eval vsim -t $SIMULATE_TIME_UNITS -lib ${LibraryName} ${LibraryUnit} ${OptionalCommands} 
+  global SimCov
+  global SimCovOptions
+  global SimCovLibUnitList
+  global SimCovTypes
+  global SimMPSoC
+
+  if {$SimCov == "On"} then {
+    # with coverage options
+    puts "vsim ${SimCovOptions} ${SimCovTypes} -t $SIMULATE_TIME_UNITS -lib ${LibraryName} ${LibraryUnit} ${SimCovLibUnitList} ${OptionalCommands}"
+     eval vsim ${SimCovOptions} ${SimCovTypes} -t $SIMULATE_TIME_UNITS -lib ${LibraryName} ${LibraryUnit} ${SimCovLibUnitList} ${OptionalCommands} 
+
+    if {[file exists ${LibraryName}.acdb]} {
+      file copy -force ${LibraryName}.acdb ${LibraryUnit}.acdb
+    };
+  } else {
+    # without coverage options (OSVVM defaults)
+    puts "vsim -t $SIMULATE_TIME_UNITS -lib ${LibraryName} ${LibraryUnit} ${OptionalCommands}"
+     eval vsim -t $SIMULATE_TIME_UNITS -lib ${LibraryName} ${LibraryUnit} ${OptionalCommands} 
+  };

Post-Processing Coverage Data

Riviera-PRO

For each design unit, the ACDB file needs processing:

  eval acdb enable -cov $SimCovTypes -verbose -du lib.spi_master
  eval acdb include -cov $SimCovTypes -verbose -du lib.spi_master

Finally, merge all ACDB files:

  eval acdb merge -verbose -cov $SimCovTypes -i *.acdb -o coverage_merged.acdb

Report merged ACDB as HTML report:

    eval acdb report -verbose -cov $SimCovTypes -html -show files -i coverage_merged.acdb -o 

Report merged ACDB as TXT report:

coverage.html
    eval acdb report -verbose -cov $SimCovTypes -txt -i coverage_merged.acdb -o coverage.txt

[JimLewis]

To address this, 2022.02 adds the following (this part already in Dev),

SetCoverageAnalyzeOptions   "String Value"
GetCoverageAnalyzeOptions   
SetCoverageAnalyzeEnable    TRUE/FALSE
GetCoverageAnalyzeEnable    
SetCoverageSimulateOptions  "String Value"
GetCoverageSimulateOptions  
SetCoverageSimulateEnable   TRUE/FALSE
GetCoverageSimulateEnable   

This is meant to be layered into the scripting. To add coverage to OSVVM tests that collects coverage on everything in the OSVVM library, but not the testbenches, do the following in a .pro file:

SetCoverageAnalyzeEnable true
include ../OsvvmLibraries
SetCoverageAnalyzeEnable false

SetCoverageSimulateEnable true
include ../OsvvmLibraries/RunAllTests.pro

# SimulateEnable needs to remain on as build finishes:
#SetCoverageSimulateEnable false

[JimLewis]

Released as part of 2022.02