verilator / verilator

Verilator open-source SystemVerilog simulator and lint system
https://verilator.org
GNU Lesser General Public License v3.0
2.58k stars 616 forks source link

[Rfc] Instance Coverage #5576

Open AndrewNolte opened 3 weeks ago

AndrewNolte commented 3 weeks ago

Instances are already tracked in coverage, but this data isn't surfaced at the moment. I put up a PR to surface it, but the points can still get crowded for large merges.

I'm proposing the addition of a "split-instances" flag when outputting with the coverage tool. Then rather than making a file for each verilog folder, it will make a file per instance:<module>.sv/<hier-path>. This will apply for both lcov and non-lcov formats.

Long term, I think it would be good to output gcovr json, which is a superset of lcov, and is more extensible for things like instances. Then downstream tools (like a vscode extension) can parse this.

wsnyder commented 3 weeks ago

I wouldn't think users would generally want separate files per instance.

  1. Some common modules may be instantiated thousands of times.
  2. The designer is usually interested in seeing if a particular module they are testing has appropriate coverage, and as such wants to look at a single file to know where things stand. But maybe I'm missing something in your desired work flow?

gcovr seems decent, but I would be concerned using the JSON format with how such a byte-verbose format would scale with large designs.

AndrewNolte commented 2 weeks ago

So the issue I'm seeing is that the per-instance points right now are all in the same file (sometimes a lot), this would essentially just be splitting those up. And there's inefficiencies in both formats, like the .dat repeats the filename, param spec, and hierarchy path for each line, when those are shared by many.

And it looks like gcov uses a zipped json to deal with compressing the keys- https://gcovr.com/en/stable/guide/gcov_parser.html

I think I'll just write an options that outputs csv from the gcovr tool, since the .dat format is pretty much just a csv.