Definition of Functional Coverage

[nishanthsmurthy24]

Hi,

From the Documentation, it was found that the functional coverage is from the ISS Simulation-flow

But, Functional Coverage is defined as the "measure of what functionalities/features of the "design" have been exercised by the tests"

And Design here refers to the RTL of a RISC-V SoC.

So, RISCV-DV technically evaluates the functional coverage only from the perspective of instructions (what instructions were covered) and not the RTL design (what parts of RTL were exercised). Is that correct?

Is there going to be any other coverage metrics (other than functional coverage) in the future?

[franAyachi]

HI!, Im agree with you, I guess that taking functional coverage from ISS does not mean we take it also from our RTL. Have you been able to use a RTL instead? thanks!

[MikeOpenHWGroup]

I worry about this approach. If your goal is to verify the RTL, then you should be collecting coverage from the RTL.

Having said that, the "UVM way" of handling this is to define a single data-type (_uvm_sequenceitem in UVM-ese), and populate in the monitor that watches retirements in the RTL and in the ISS monitor. In this way it is possible to collect functional coverage from environments that do not have RTL (useful for bring-up testing and seeing how good your test-programs are.

Have you looked at CORE-V Wally? The "Wally" core has an excellent functional coverage model and follows a strategy similar to the above.

As for other coverage, I strongly recommend code coverage of the RTL. Of course, that is a tool issue, not a riscv-dv issue.

[franAyachi]

I worry about this approach. If your goal is to verify the RTL, then you should be collecting coverage from the RTL.

Having said that, the "UVM way" of handling this is to define a single data-type (_uvm_sequenceitem in UVM-ese), and populate in the monitor that watches retirements in the RTL and in the ISS monitor. In this way it is possible to collect functional coverage from environments that do not have RTL (useful for bring-up testing and seeing how good your test-programs are.

Have you looked at CORE-V Wally? The "Wally" core has an excellent functional coverage model and follows a strategy similar to the above.

As for other coverage, I strongly recommend code coverage of the RTL. Of course, that is a tool issue, not a riscv-dv issue.

Hi @MikeOpenHWGroup, thanks for your answer like always!

I completely agree with you. I am a new user of RISC-V-related technologies and am particularly interested in functional coverage. That's why I wanted to explore what RISC-V-DV offers in this area as well. As you mentioned, cores like Wally utilize licensed tools (ImperasDV-riscvISAcov) to generate SystemVerilog files with ISA-level coverage.

I am interested in reusing existing code because manually implementing functional coverage for RISC-V can be quite labor-intensive. However, I currently don't have access to ImperasDV, and I would also need to define a tracer to comply with the RVVI interface.

That being said, I noticed the following in RISCV-DV's src/riscv_instr_cover_group.sv file: // By default the coverage model run with instruction trace from ISS simulation. When simulating // with ISS, certain covergroups like debug/interrupt could not be hit as these are not simulated // with ISS. You can turn off this mode by adding +iss_mode=0 if you use the instruction trace // from RTL simulation. bit iss_mode = 1'b1;

and in the coverage generation script cov.py: if argv.core: """If functional coverage is being collected from an RTL core implementation, the flow assumes that the core's trace logs have already been converted to CSV files by the post_compare step of the flow. """ trace_log = ("{}/{}_trace_log".format(out, argv.core)) run_cmd("find {} -name \"*.csv\" | sort > {}".format(argv.dir, trace_log))

Thank you very much for your help!

[MikeOpenHWGroup]

This is becoming a bit of a discussion, rather than an Issue resolution, and I am not sure the code-owners of this repo want to use Issues for such discussions. As such, I'll try to answer your questions above and then will not update this Issue further. If you like, please reach out to me directly and we can debate the topic off-line.

As you mentioned, cores like Wally utilize licensed tools (ImperasDV-riscvISAcov) to generate SystemVerilog files with ISA-level coverage.

That is partially true. Wally does use the ISA coverage model from Synopsys/Imperas, but is also developing its own open-source version. It should be available on the repo. There is also ISACOV available here. (Note, ISACOV covers RV32IMC only.)

However, I currently don't have access to ImperasDV

You do not need ImperasDV if you collect functional coverage from the RTL's tracer.

I would also need to define a tracer to comply with the RVVI interface.

You will need a Tracer anyway, you might as well use a good one. 😛

That being said, I noticed the following in RISCV-DV's...

I do not recommend using the RISCV-DV approach for collecting coverage. Your environment will be much easier to use and maintain if you place on Monitor on the Tracer and use that Monitor to unpack Tracer transactions into the _uvm_sequenceitem used by your coverage model and then sample() it directly.