search

bazelbuild / bazel

a fast, scalable, multi-language and extensible build system
https://bazel.build
Apache License 2.0
23.21k stars 4.06k forks source link

No coverage report found for python bazel coverage #18538

Closed nlou9 closed 1 year ago

nlou9 commented 1 year ago

Description of the bug:

% bazel coverage //tests/ci/scripts:test_ci                                                                         
INFO: Invocation ID: b7d67126-0098-4bf0-8402-b1ef84d7472b
WARNING: Streamed test output requested. All tests will be run locally, without sharding, one at a time
INFO: Using default value for --instrumentation_filter: "^//tests/ci/scripts[/:]".
INFO: Override the above default with --instrumentation_filter
INFO: Analyzed target //tests/ci/scripts:test_ci (0 packages loaded, 146 targets configured).
INFO: Found 1 test target...
+ [[ -z bazel-out/darwin-fastbuild/bin/tests/ci/scripts/test_ci.instrumented_files ]]
+ export ROOT=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools
+ ROOT=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools
+ [[ bazel-out/darwin-fastbuild/bin/tests/ci/scripts/test_ci.instrumented_files != /* ]]
+ export COVERAGE_MANIFEST=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/bin/tests/ci/scripts/test_ci.instrumented_files
+ COVERAGE_MANIFEST=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/bin/tests/ci/scripts/test_ci.instrumented_files
+ export COVERAGE_DIR=bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/_coverage
+ COVERAGE_DIR=bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/_coverage
+ [[ bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/_coverage == /private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools* ]]
+ COVERAGE_DIR=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/_coverage
+ mkdir -p /private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/_coverage
+ COVERAGE_OUTPUT_FILE=bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/coverage.dat
+ [[ bazel-out/darwin-fastbuild/testlogs/testsci/scripts/test_ci/coverage.dat == /private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools* ]]
+ COVERAGE_OUTPUT_FILE=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/coverage.dat
+ export JAVA_COVERAGE_FILE=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/_coverage/jvcov.dat
+ JAVA_COVERAGE_FILE=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/_coverage/jvcov.dat
+ export COVERAGE=1
+ COVERAGE=1
+ export BULK_COVERAGE_RUN=1
+ BULK_COVERAGE_RUN=1
+ [[ -z '' ]]
+ export GCOV_PREFIX_STRIP=3
+ GCOV_PREFIX_STRIP=3
+ export GCOV_PREFIX=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/_coverage
+ GCOV_PREFIX=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/_coverage
+ export LLVM_PROFILE_FILE=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/_coverage/%h-%p-%m.profraw
+ LLVM_PROFILE_FILE=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/_coverage/%h-%p-%m.profraw
+ [[ -n '' ]]
+ [[ ! -z '' ]]
+ [[ 0 == \0 ]]
+ cd /private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/bin/tests/ci/scripts/test_ci.runfiles/ci-tools
+ [[ 0 == \0 ]]
+ touch /private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/coverage.dat
+ /private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/bin/tests/ci/scripts/test_ci.runfiles/ci-tools/tests/ci/scripts/test_ci '--junit-xml=$XML_OUTPUT_FILE' --ignore=external
Using toolchain coverage_tool 'python3_9_x86_64-apple-darwin_coverage/coverage/__main__.py'
Coverage entrypoint: /private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/bin/tests/ci/scripts/test_ci.runfiles/python3_9_x86_64-apple-darwin_coverage/coverage/__main__.py
============================= test session starts ==============================
platform darwin -- Python 3.9.16, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/bin/tests/ci/scripts/test_ci.runfiles/ci-tools
plugins: common-tools-1.2.1681
collected 27 items

tests/ci/scripts/test_ci.py ...........................        [100%]

=============================== warnings summary ===============================
../pip_glob2/site-packages/glob2/fnmatch.py:141
  /private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/bin/tests/ci/scripts/test_ci.runfiles/pip_glob2/site-packages/glob2/fnmatch.py:141: DeprecationWarning: invalid escape sequence \Z
    return '(?ms)' + res + '\Z'

-- Docs: https://docs.pytest.org/en/stable/warnings.html
- generated xml file: /private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/test.xml -
======================== 27 passed, 1 warning in 2.46s =========================
Converting coveragepy database to lcov: /private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/_coverage/pylcov.dat
Wrote LCOV report to /private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/_coverage/pylcov.dat
+ TEST_STATUS=0
+ [[ 0 -ne 0 ]]
+ [[ 0 == \1 ]]
+ [[ 0 == \1 ]]
+ cd /private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools
+ [[ -n external/bazel_tools/tools/test/collect_cc_coverage.sh ]]
+ eval external/bazel_tools/tools/test/collect_cc_coverage.sh
++ external/bazel_tools/tools/test/collect_cc_coverage.sh
+ main
+ init_gcov
+ GCOV=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/_coverage/gcov
+ '[' '!' -f '' ']'
+ echo 'GCov does not exist at the given path: '\'''\'''
GCov does not exist at the given path: ''
+ exit 1
+ [[ -z bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/remote_coverage_tools/Main ]]
+ for name in '"$LCOV_MERGER"'
+ [[ ! -e bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/remote_coverage_tools/Main ]]
+ [[ 0 == \1 ]]
+ LCOV_MERGER_CMD='bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/remote_coverage_tools/Main --coverage_dir=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/_coverage   --output_file=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/coverage.dat   --filter_sources=/usr/bin/.+   --filter_sources=/usr/lib/.+   --filter_sources=/usr/include.+   --filter_sources=/Applications/.+   --source_file_manifest=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/bin/tests/ci/scripts/test_ci.instrumented_files'
+ [[ -n '' ]]
+ [[ -n '' ]]
+ JAVA_RUNFILES=
+ exec bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/remote_coverage_tools/Main --coverage_dir=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/_coverage --output_file=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/coverage.dat --filter_sources=/usr/bin/.+ --filter_sources=/usr/lib/.+ --filter_sources=/usr/include.+ --filter_sources=/Applications/.+ --source_file_manifest=/private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/bin/tests/ci/scripts/test_ci.instrumented_files
May 20, 2023 12:16:09 AM com.google.devtools.coverageoutputgenerator.Main getTracefiles
INFO: Found 1 tracefiles.
May 20, 2023 12:16:09 AM com.google.devtools.coverageoutputgenerator.Main parseFilesSequentially
INFO: Parsing file /private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/sandbox/darwin-sandbox/6/execroot/ci-tools/bazel-out/darwin-fastbuild/testlogs/tests/ci/scripts/test_ci/_coverage/pylcov.dat
May 20, 2023 12:16:09 AM com.google.devtools.coverageoutputgenerator.Main getGcovInfoFiles
INFO: No gcov info file found.
May 20, 2023 12:16:09 AM com.google.devtools.coverageoutputgenerator.Main getGcovJsonInfoFiles
INFO: No gcov json file found.
May 20, 2023 12:16:09 AM com.google.devtools.coverageoutputgenerator.Main getProfdataFileOrNull
INFO: No .profdata file found.
May 20, 2023 12:16:09 AM com.google.devtools.coverageoutputgenerator.Main runWithArgs
WARNING: There was no coverage found.
INFO: LCOV coverage report is located at /private/var/tmp/_bazel_nlou/94d2fc5507934f261f7423a31b5925d0/execroot/ci-tools/bazel-out/_coverage/_coverage_report.dat
 and execpath is bazel-out/_coverage/_coverage_report.dat
Target //tests/ci/scripts:test_ci up-to-date:
  bazel-bin/tests/ci/scripts/test_ci
INFO: Elapsed time: 0.838s, Critical Path: 0.36s
INFO: 2 processes: 1 disk cache hit, 1 internal.
INFO: Build completed successfully, 2 total actions
//tests/ci/scripts:test_ci                            (cached) PASSED in 13.4s

Executed 0 out of 1 test: 1 test passes.

.bazelrc

coverage --action_env=VERBOSE_COVERAGE=true
coverage --combined_report=lcov
coverage --verbose_failures
coverage --instrument_test_targets=false
coverage --test_tag_filters=coverage
coverage --test_output=streamed
coverage --sandbox_debug

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

No response

Which operating system are you running Bazel on?

darwin-x86_64

What is the output of bazel info release?

% bazel info release INFO: Invocation ID: 9c0fa5ad-bf7a-46b3-a192-4b23f4f9a4d0 release 6.1.2

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Is this a regression? If yes, please try to identify the Bazel commit where the bug was introduced.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

Pavank1992 commented 1 year ago

Hi @nlou9, Could you please help us to reproduce this ?

nlou9 commented 1 year ago

I just realized that the default instrumentation filter is "^//tests/ci/scripts[/:]" for py_test. After I set it into --instrumentation_filter="^//", it works. Why does it set the default instrumentation filter as tests directory for this py_test? 

py_test(
    name = "test_ci",
    srcs = [
        "__test__.py",
        "test_ci.py",
    ],
    data = ["//:config"],
    main = "__test__.py",
    deps = [
        "//ci/scripts",
        "//tests/data",
        requirement("click"),
        requirement("mock"),
        requirement("pytest"),
    ],
)
c-mita commented 1 year ago

When no instrumentation filter is provided, one is heuristically calculated.

Broadly speaking, the calculated filter will be:

Since //tests/ci/scripts:test_ci doesn't match any of those patterns, nothing is changed.