Features and workflows to run regression, PyUVM, cocotb tests and present results in GitHub Pages environment are introduced in this PR. In particular, the following changes were made:
VM_COVERAGE macro is added to collect coverage data from Verilator simulations
Environment variable COVERAGE is defined in the $RV_ROOT/tools/Makefile to pass coverage type argument to the Verilator.
To the same effect, addoption fixture is used in configuration of test_pyuvm
cocotb is now a submodule
main pipeline of CI is reworked, please find a more detailed description below
Workflow structure
The workflow is defined in the ci.yml file and is a top-level view for the following workflows:
Build Verilator
Test Regression
Test Verification
Report Coverage
Report Pytest
Publish to GH Pages
The workflows are executed in order: Build -> Test -> Report -> Publish. BASH scripts found under .github/scripts/*.sh are used by workflows to perform required tasks.
Operation of the pipeline
Selected version of Verilator is compiled and cached. This part of the workflow is meant to be run nightly. If the Verilator is available in cache, it is restored, which greatly decreases execution time of the pipeline.
Verification and regression tests are run in parallel. It is expected that verification tests will produce pytest-html reports. It is expected that all Verilator simulations will produce coverage data files.
Produced data is fed into Report Pytest and Report Coverage jobs, which convert and consolidate data into coherent HTML views, wrapped in a top-level page and subsequent "Coverage Dashboard" and "Verification Dashboard".
The last job, Publish to GH pages is responsible for pushing collected data to the gh-pages branch of the repository to publish the results. Link to deployments on fork
Introduction
Features and workflows to run regression,
PyUVM
,cocotb
tests and present results in GitHub Pages environment are introduced in this PR. In particular, the following changes were made:COVERAGE
is defined in the$RV_ROOT/tools/Makefile
to pass coverage type argument to the Verilator.addoption
fixture is used in configuration oftest_pyuvm
Workflow structure
The workflow is defined in the
ci.yml
file and is a top-level view for the following workflows:.github/scripts/*.sh
are used by workflows to perform required tasks.Operation of the pipeline
Report Pytest
andReport Coverage
jobs, which convert and consolidate data into coherent HTML views, wrapped in a top-level page and subsequent "Coverage Dashboard" and "Verification Dashboard".Publish to GH pages
is responsible for pushing collected data to thegh-pages
branch of the repository to publish the results. Link to deployments on fork