CP-41819: Py3: Collect, check and report code coverage from Python3 unit test runs

[bernhardkaindl]

Collect, check and report code coverage from Python3 unit test runs:

• Add .coveragerc and mark not covered branches

• Collect and check minium and diff code coverage in pre-commit

• For GitHub CI:

  • Add the Pytest Coverage Comment action, it shows:

  • Number of Python3 pytest tests executed, skipped, failures, errors and duration

  • When the title above the table is expanded:

    • The complete list of all not covered lines from the Python3 test run is shown
    • These reported lines have links to the source code

  • Add coverage upload to https://codecov.io, which once merged:

  • Provide a link to Codecov code coverage analysis, with the changes to the coverage

  • The Codecov comment will report if all changes lines are not covered by the unit test.

Sample of how such Codecov comment will look like:

---

All modified and coverable lines are covered by tests ✅

Comparison is base (1fe372c) 50.63% compared to head (b85dc26) 51.29%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master      #16      +/-   ##
==========================================
+ Coverage   50.63%   51.29%   +0.65%     
==========================================
  Files          13       13              
  Lines        1815     1817       +2     
==========================================
+ Hits          919      932      +13     
+ Misses        896      885      -11

[github-actions[bot]]

https://github.com/marketplace/actions/pytest-coverage-comment

File	Stmts	Miss	Cover	Missing
__init__.py	0	0	100%
conftest.py	30	0	100%
test_dir_list.py	11	0	100%
test_fs_funcs.py	8	0	100%
test_load_plugins.py	7	0	100%
test_output.py	43	0	100%
test_process_output.py	8	0	100%
test_xapidb_filter.py	17	0	100%
xen-bugtool	1524	729	52%	95, 100–101, 410, 490–491, 493–494, 511, 517, 540, 551–552, 555–556, 598, 602–604, 608, 610–614, 616–619, 628–630, 633, 652–653, 656, 658–662, 664–665, 668–669, 673–674, 676–677, 681–684, 686–689, 692–694, 696–697, 701–703, 706–707, 716, 718, 720–723, 725–727, 729–730, 734–735, 737, 739–745, 749–750, 752–756, 761–769, 771–773, 775–777, 779–780, 782, 797, 799–803, 805–806, 808, 810, 812–835, 838–842, 844, 846–867, 870–871, 873, 875, 878, 880–889, 891, 893–896, 898–908, 910–926, 928–932, 934–940, 943–972, 974–984, 986–989, 991–1011, 1013–1016, 1018, 1020, 1022–1023, 1025–1026, 1028, 1043–1044, 1046–1047, 1051–1052, 1054–1055, 1057, 1059, 1061–1062, 1064–1065, 1067–1076, 1079–1085, 1088–1089, 1091–1092, 1101–1102, 1104–1105, 1109, 1131–1134, 1136–1140, 1142–1149, 1151, 1153–1156, 1158, 1161–1163, 1165–1171, 1174–1176, 1180–1181, 1183–1184, 1186–1188, 1191–1192, 1195, 1197–1198, 1200, 1202, 1204–1205, 1209–1212, 1215, 1218, 1221–1223, 1226–1230, 1233–1234, 1237–1240, 1246–1252, 1255–1259, 1261–1277, 1280–1288, 1291–1292, 1294, 1296–1297, 1299–1307, 1310, 1312–1315, 1317, 1319–1320, 1354, 1382, 1397, 1405–1406, 1410–1412, 1414, 1417, 1419–1421, 1423–1427, 1429, 1433, 1436, 1439–1445, 1447, 1449–1453, 1459–1461, 1463–1468, 1471–1475, 1479–1480, 1482, 1499–1500, 1502, 1504–1505, 1507–1509, 1511–1518, 1520–1525, 1529–1530, 1532, 1534, 1536, 1552–1553, 1555, 1557–1559, 1562–1563, 1565, 1567–1569, 1572–1573, 1575, 1577–1580, 1583–1593, 1596–1600, 1610, 1622, 1632, 1636, 1647, 1671–1673, 1685–1688, 1696, 1711, 1715–1719, 1724–1726, 1739, 1745, 1755–1761, 1778, 1799–1802, 1804–1805, 1817, 1877, 1896, 1898, 1900, 1903–1911, 1914–1915, 1918–1925, 1927–1928, 1930, 1934–1935, 1939, 1943–1945, 1949–1950, 1953, 1957, 1962, 1968–1969, 1971, 1975, 1977–1980, 1983–1993, 1997–1998, 2002, 2004, 2027–2036, 2039, 2089–2091, 2094–2096, 2108, 2127–2129, 2169–2173, 2180, 2182–2185, 2189, 2203
TOTAL	1648	729	55%

Tests	Skipped	Failures	Errors	Time
10	0 :zzz:	0 :x:	0 :fire:	0.737s :stopwatch:

[codecov-commenter]

Welcome to Codecov :tada:

Once merged to your default branch, Codecov will compare your coverage reports and display the results in this comment.

Thanks for integrating Codecov - We've got you covered :open_umbrella: