onsi / ginkgo

A Modern Testing Framework for Go
http://onsi.github.io/ginkgo/
MIT License
8.29k stars 654 forks source link

Could not process Coverprofile for multi-module project #1323

Open austince opened 9 months ago

austince commented 9 months ago

I'm getting an error when trying to run tests with coverage for a module that exists in a monorepo with many other modules.

Ginkgo version: v2.13.2 go version go1.21.0 darwin/arm64

The project structure looks like this:

imports.go # dummy file that imports the submodules
go.mod # dummy module, with local replace directives

submodule-1/
  go.mod
  pkg/
     pkg.go
     pkg_test.go

submodule-2/
  go.mod

Project root go.mod

module multimod

go 1.21

replace (
    multimod/submodule-1 => ./submodule-1
        multimod/submodule-2 => ./submodule-2
)

require (
    multimod/submodule-1 v0.0.0-00010101000000-000000000000
    multimod/submodule-2 v0.0.0-00010101000000-000000000000
)

submodule-1 go.mod

module multimod/submodule-1

go 1.21

replace (
        multimod/submodule-2 => ../submodule-2
)

require (
    multimod/submodule-2 v0.0.0-00010101000000-000000000000
)

From the project root directory:

> ginkgo run -race -r --cover --coverprofile coverage.txt "./submodule-1"
...
PASS
coverage: 47.3% of statements
...
PASS
coverage: 81.8% of statements
...
ginkgo run failed
  could not finalize profiles:
  Could not process Coverprofile coverage.txt: exit status 1

Results in a coverage.txt file in the project root like:

mode: atomic
multimod/submodule-1/pkg/pkg.go:15.58,16.25 1 0
multimod/submodule-1/pkg/pkg.go:16.25,18.3 1 0
multimod/submodule-1/pkg/pkg.go:20.2,20.55 1 0
multimod/submodule-1/pkg/pkg.go:20.55,21.34 1 0

Is there anything I can do to get more information on why the cover profile could not be processed?

onsi commented 8 months ago

hey sorry for the delay. ginkgo is running go tool cover -func coverage.txt but not showing you the output if it exists with a non-zero exit code. i've fixed that now so you'll at least get the output (will cut a release soon) - but you can also run the command yourself to see what it returns and try to debug this further. it may be that how Ginkgo is merging the coverage files is causing issues.

ralf-cestusio commented 6 months ago

I have seen something similar in my case (But there the problem stems from the fact that ginkgo executes tests over module boundaries. see also https://github.com/onsi/ginkgo/issues/801

So in my case having a base module and a module in folder v2

after the ginkgo run have a coverprofile in the base folder which contains information about tests in submodules

mode: atomic code.ops.wombat.org/api/mod/pkg/test/print.go:5.14,7.2 1 0 code.ops.wombat.org/api/mod/v2/pkg/test/print.go:5.14,7.2 1 0

So when go tool cover runs it tries to resolve code.ops.wombat.org/api/mod/v2 in the base module which naturally fails because for me that module has no dependency on the v2 version

go tool cover -func cover.coverprofile code.ops.wombat.org/api/mod/pkg/test/print.go:5: Print 0.0% cover: no required module provides package code.ops.wombat.org/api/mod/v2/pkg/test; to add it: go get code.ops.wombat.org/api/mod/v2/pkg/test