cmd/go: inconsistent coverage when using -coverprofile with packages without tests

NarcisDavins commented 6 days ago

Go version

go version go1.23.3 linux/amd64

Output of `go env` in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/root/.cache/go-build'
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.23.3'
GODEBUG=''
GOTELEMETRY='local'
GOTELEMETRYDIR='/root/.config/go/telemetry'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2962845657=/tmp/go-build -gno-record-gcc-switches'

What did you do?

Given the following project, with pkg1 not containing any test, and pkg2 with 1 test that is calling pkg1, when using go test with the -coverprofile option, the cover file generated contains pkg1 as covered

$->> cov $ tree
.
├── go.mod
├── pkg1
│   └── file.go
└── pkg2
    ├── file.go
    └── file_test.go

3 directories, 4 files
$->> cov $ cat go.mod 
module cov
$->> cov $ cat pkg1/file.go 
package pkg1

func DoSomething() bool {
    return true;
}
$->> cov $ cat pkg2/file.go 
package pkg2

func DoSomething() bool {
    return true;
}
$->> cov $ cat pkg2/file_test.go 
package pkg2

import "testing"
import "cov/pkg1"

func TestSmth(t * testing.T) {
  pkg1.DoSomething()
  DoSomething()
}

What did you see happen?

Running go test with -coverprofile the output correctly shows 0% coverage on the package without tests, but the generated file info is marking pkg1 as covered

$->> cov $ docker run --rm -it -v .:/code -w /code golang:1.23.3 go test ./... -coverprofile cov.out
    cov/pkg1        coverage: 0.0% of statements
ok      cov/pkg2    0.003s  coverage: 100.0% of statements
$->> cov $ cat cov.out 
mode: set
cov/pkg1/file.go:3.25,5.2 1 0
cov/pkg1/file.go:3.25,5.2 1 1
cov/pkg2/file.go:3.25,5.2 1 1

What did you expect to see?

pkg1 should not be marked as covered in the cover file, as it does not have any test and I'm not using other params such as -coverpkg.

using GOEXPERIMENT=nocoverageredesign is currently a workaround we are using, but would stop being an option after: https://github.com/golang/go/issues/55953.

Another workaround would be to generate empty test files on all packages, but as far as I can tell, this shouldn't be the expected behavior.

$->> cov $ echo "package pkg1" >> pkg1/file_test.go
$->> cov $ docker run --rm -it -v .:/code -w /code golang:1.23.3 go test ./... -coverprofile cov.out
ok      cov/pkg1    0.003s  coverage: 0.0% of statements [no tests to run]
ok      cov/pkg2    0.003s  coverage: 100.0% of statements
$->> cov $ cat cov.out 
mode: set
cov/pkg1/file.go:3.25,5.2 1 0
cov/pkg2/file.go:3.25,5.2 1 1

gabyhelp commented 6 days ago

Related Issues and Documentation

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

dr2chase commented 5 days ago

@thanm we still list you as owner, maybe you still have background on this? otherwise, @golang/compiler

thanm commented 5 days ago

Hi @NarcisDavins you wrote:

pkg1 should not be marked as covered in the cover file, as it does not have any test and I'm not using other params such as -coverpkg.

I'm confused about what you're asking here. In the contents of cov.out there is this line:

cov/pkg1/file.go:3.25,5.2 1 0

That indicates that package pkg1 has 1 statement and that it is not covered. When you say "marked as covered" do you mean that pkg should not be mentioned at all?

NarcisDavins commented 5 days ago

Hi @NarcisDavins you wrote:

pkg1 should not be marked as covered in the cover file, as it does not have any test and I'm not using other params such as -coverpkg.

I'm confused about what you're asking here. In the contents of cov.out there is this line:
cov/pkg1/file.go:3.25,5.2 1 0
That indicates that package pkg1 has 1 statement and that it is not covered. When you say "marked as covered" do you mean that pkg should not be mentioned at all?

@thanm pkg1 appears twice, the line you copied where it indeed shows as not covered, and the following one where it appears as covered. I'll paste again the "What did you see happen?" part of the issue:

$->> cov $ docker run --rm -it -v .:/code -w /code golang:1.23.3 go test ./... -coverprofile cov.out
    cov/pkg1        coverage: 0.0% of statements
ok      cov/pkg2    0.003s  coverage: 100.0% of statements
$->> cov $ cat cov.out 
mode: set
cov/pkg1/file.go:3.25,5.2 1 0
cov/pkg1/file.go:3.25,5.2 1 1
cov/pkg2/file.go:3.25,5.2 1 1

Edit: that second line where it appears as covered is the one that should not be there.

thanm commented 4 days ago

Right, of course. Somehow didn't pick up on that the first time. Thanks for the clarification.

gopherbot commented 20 hours ago

Change https://go.dev/cl/627315 mentions this issue: internal/coverage: refactor EmitTextual in preparation for bugfix

gopherbot commented 20 hours ago

Change https://go.dev/cl/627316 mentions this issue: internal/coverage: pass selected package set to EmitTextual

golang / go