Improved code coverage and tests refactoring

[lbartoletti]

QGIS Enhancement: Improved code coverage and tests refactoring

Date 2021/03/22

Authors

• Loïc Bartoletti (@lbartoletti)
• Sébastien Peillet (@SebastienPeillet)

Contact loic dot bartoletti at oslandia dot com

maintainer @lbartoletti

Version QGIS 3.22 / 3.24

Summary

QGIS, and in particular its API, is well covered by unit tests. However, we noticed during the last developments that there is still blind spots in the test coverage of these classes, despite of the existing ones. Moreover, some may be redundant (and cost build and CI time), some are different depending on the environment.

Proposed Solution

Missing tests

Write the missing tests on critical part and fix the ones that could be wrong as it could be encountered in the past (i.e the Wkt parser). As a first step, we propose to focus on geometry classes and map tools.

Refactoring

We also propose a slight refactoring and in particular to split into several parts some files that are too large (I think about you in particular testqgsgeometry.cpp with your 18713 lines...).

Python tests

We have had, and still have, differences results between the C++ API and the python API. We would like to add the equivalent of c++ tests on the python binding.

CI/Coverage

There is currently code in cmake to generate code coverage, we will investigate the possibility of generating these reports to view and measure the evolution of code coverage.

Votes

(required)

[anitagraser]

Please add the Grant-2021 label, thanks!

[jmarsac]

👍

[lbartoletti]

For information, I have created a dedicated repository for testing: https://github.com/lbartoletti/qgis_coverage/tree/coverage

For now, the result is from a separate build on one of my machines and pushed in this repository. We plan to switch the automation to a public CI.

These reports are built from the coverage of the geometry tests: ${MAKE} qgis_coverage -j${NB_PROC} CMD_ARGS="-R qgis_*geometry*"

There is two reports available:

• https://lbartoletti.github.io/qgis_coverage/2021-03-26/coverage/core/geometry/index.html
• https://lbartoletti.github.io/qgis_coverage/2021-04-10/coverage/core/geometry/index.html (containing https://github.com/qgis/QGIS/pull/42764)

[nyalldawson]

@lbartoletti i don’t think your test run is complete — eg the box3d class is well covered by the tests in https://github.com/qgis/QGIS/blob/master/tests/src/python/test_qgsbox3d.py yet the report states there is no coverage

[nyalldawson]

Ah indeed, the Rexeg you’ve used to control which tests are run is excluding all the python geometry tests, and the specialised tests for the rectangle and box classes.