Coverage reporting

[mkurc-ant]

Introduction

Features and workflows to run regression, PyUVM, cocotb tests and present results in GitHub Pages environment are introduced in this PR. In particular, the following changes were made:

• VM_COVERAGE macro is added to collect coverage data from Verilator simulations
• Environment variable COVERAGE is defined in the $RV_ROOT/tools/Makefile to pass coverage type argument to the Verilator.
• To the same effect, addoption fixture is used in configuration of test_pyuvm
• cocotb is now a submodule
• main pipeline of CI is reworked, please find a more detailed description below

  Workflow structure

  The workflow is defined in the ci.yml file and is a top-level view for the following workflows:

• Build Verilator
• Test Regression
• Test Verification
• Report Coverage
• Report Pytest
• Publish to GH Pages The workflows are executed in order: Build -> Test -> Report -> Publish. BASH scripts found under .github/scripts/*.sh are used by workflows to perform required tasks.

  Operation of the pipeline

  1. Selected version of Verilator is compiled and cached. This part of the workflow is meant to be run nightly. If the Verilator is available in cache, it is restored, which greatly decreases execution time of the pipeline.
  2. Verification and regression tests are run in parallel. It is expected that verification tests will produce pytest-html reports. It is expected that all Verilator simulations will produce coverage data files.
  3. Produced data is fed into Report Pytest and Report Coverage jobs, which convert and consolidate data into coherent HTML views, wrapped in a top-level page and subsequent "Coverage Dashboard" and "Verification Dashboard".
  4. The last job, Publish to GH pages is responsible for pushing collected data to the gh-pages branch of the repository to publish the results. Link to deployments on fork