The code coverage displayed on the front page of the project is currently 27%, but this is a misleading number. It is the percentage of Python code exercised in the dragonphy directory, despite the fact that the design is mostly described in Verilog and TCL code. Plus, there is a bunch of code in the dragonphy directory that isn't used in generating the design, while there is additional mflowgen Python code that is not included in the coverage statistic.
We should fix these problems by adjusting the scope of the code that is included: count only related Python code, and count Verilog and TCL. Generating coverage for Verilog is straightforward, but integrating it into the codecov analysis may not be, because Verilog is not currently one of the supported languages. TCL is even more unknown, since I'm not sure if synthesis and PnR tools can produce a coverage output (TCL isn't supported by codecov, either).
This might seem like a cosmetic issue, but the purpose of coverage analysis is to find gaps in testing, so if we're not considering the right source code, we don't really learn anything from the analysis.
The code coverage displayed on the front page of the project is currently 27%, but this is a misleading number. It is the percentage of Python code exercised in the
dragonphydirectory, despite the fact that the design is mostly described in Verilog and TCL code. Plus, there is a bunch of code in thedragonphydirectory that isn't used in generating the design, while there is additionalmflowgenPython code that is not included in the coverage statistic.We should fix these problems by adjusting the scope of the code that is included: count only related Python code, and count Verilog and TCL. Generating coverage for Verilog is straightforward, but integrating it into the
codecovanalysis may not be, because Verilog is not currently one of the supported languages. TCL is even more unknown, since I'm not sure if synthesis and PnR tools can produce a coverage output (TCL isn't supported bycodecov, either).This might seem like a cosmetic issue, but the purpose of coverage analysis is to find gaps in testing, so if we're not considering the right source code, we don't really learn anything from the analysis.