search

sematic-ai / sematic

An open-source ML pipeline development platform
Other
972 stars 59 forks source link

How to generate coverage report #1068

Closed allenwang-git closed 1 year ago

allenwang-git commented 1 year ago

I use bazel test //sematic/tests/... command after modify .bazrlrc to coverage --test_tag_filters=cov and got such error:

Executing tests from //sematic/tests:test_user_settings_coverage
-----------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/allen-wang/.cache/bazel/_bazel_allen-wang/30e441248d5657e95dfb991e05083fbc/sandbox/linux-sandbox/56/execroot/sematic/bazel-out/k8-fastbuild/bin/sematic/tests/test_user_settings_coverage.runfiles/sematic/sematic/tests/test_user_settings_coverage", line 559, in <module>
    Main()
  File "/home/allen-wang/.cache/bazel/_bazel_allen-wang/30e441248d5657e95dfb991e05083fbc/sandbox/linux-sandbox/56/execroot/sematic/bazel-out/k8-fastbuild/bin/sematic/tests/test_user_settings_coverage.runfiles/sematic/sematic/tests/test_user_settings_coverage", line 512, in Main
    raise EnvironmentError(
OSError: Python coverage tool '/home/allen-wang/.cache/bazel/_bazel_allen-wang/30e441248d5657e95dfb991e05083fbc/sandbox/linux-sandbox/56/execroot/sematic/bazel-out/k8-fastbuild/bin/sematic/tests/test_user_settings_coverage.runfiles/external/pip_dependencies_coverage/rules_python_wheel_entry_point_coverage' not found. Try running with VERBOSE_COVERAGE=1 to collect more information.
--
Coverage runner: Not collecting coverage for failed test.
The following commands failed with status 1
/home/allen-wang/.cache/bazel/_bazel_allen-wang/30e441248d5657e95dfb991e05083fbc/sandbox/linux-sandbox/56/execroot/sematic/bazel-out/k8-fastbuild/bin/sematic/tests/test_user_settings_coverage.runfiles/sematic/sematic/tests/test_user_settings_coverage sematic/tests/test_user_settings.py

if I change it to test --test_tag_filters=cov than it can run //sematic/tests:test_*_coverage targets successfully but only empty coverage.dat files generated. I'm wondering what's the correct way to generate coverage report for the tests.

tscurtu commented 1 year ago

Hello, @allenwang-git! He had disabled the coverage tests a while back because they were flaky due to infrastructure issues, and because we didn't actually use the reports. Please see https://github.com/sematic-ai/sematic/pull/496.