coverage data with godog

[montera82]

just like go test where i could do go test -coverprofile=coverage.txt

any ideas how this could be achieved with godog ?

i.e something like godog --coverprofile=coverage.txt

basically i need to output some kind of coverage data with godog

[l3pp4rd]

Hi, you need to integrate godog with go test and run your tests as usual with go test the coverage will work as expected. see how to configure it in readme, the godog command is optional, but it will still work with go test integration

[montera82]

thanks @l3pp4rd , i setup TestMain in my project root as follows:

file ./main_test.go

package mapper

import (
    "github.com/DATA-DOG/godog"
    "github.com/mycompany/mapper/config"
    b "github.com/mycompany/mapper/features/bootstrap"
    log "github.com/sirupsen/logrus"
    "testing"
    "os"
)

var (
    URL = "http://localhost"
)

func TestMain(m *testing.M) {
    status := godog.RunWithOptions("godog", func(s *godog.Suite) {
        conf, err := config.LoadEnv()
        if err != nil {
            log.Fatal(err)
        }

        b.FeatureContext(s, URL)
        b.RecipesContext(s, conf.BrokerDSN)

    }, godog.Options{
        Format: "pretty",
    })

    if st := m.Run(); st > status {
        status = st
    }

    os.Exit(status)
}

and then when i run

/go/src/github.com/mycompany/mapper # go test -covermode=atomic -coverprofile=cover.txt main_test.go 
ok      command-line-arguments  0.502s  coverage: 0.0% of statements [no tests to run]

and i get an empty cover.txt file created with content mode: atomic

even though i noticed it successfully ran the godog tests i have in./features/* folder, why does it say [no tests to run] and why do i have no reasonable coverage data in cover.txt?

[l3pp4rd]

why do you specify main_test.go as an argument? go test -covermode=atomic -coverprofile=cover.txt main_test.go run it with: go test -covermode=atomic -coverprofile=cover.txt if I run it in godog library it properly shows the coverage, maybe you have another TestMain function somewhere? what go version?

[montera82]

hi @l3pp4rd i tried with go test -covermode=atomic -coverprofile=cover.txt eventhough in stdin, i see the runing of godog tests , it still does not record any coverage to cover.txt only thing in the .txt file is a line with text mode: atomic

confirmed only one TestMain func in the project...

/go/src/github.com/mycompany/mapper # go version
go version go1.8.3 linux/amd64

i got enough godog tests to aleast see 5% coverage, see last portion of the output i now get

7 scenarios (7 passed)
31 steps (31 passed)
549.2454ms
testing: warning: no tests to run
PASS
coverage: 0.0% of statements
ok      github.com/mycompany/mapper        0.584s

[montera82]

also see my env...(btw i am running inside docker)

LOG_LEVEL=debug
DEBUG=true
HOSTNAME=e6c97f5907ac
SHLVL=1
HOME=/root
PORT=80
STATSD_DSN=noop://
SERVICE_NAME=mapper
TERM=xterm
PATH=/go/bin:/usr/local/go/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
APP_DIR=/go/src/github.com/mycompany/mapper
GOPATH=/go
PWD=/go/src/github.com/mycompany/mapper
GOLANG_VERSION=1.8.3

[l3pp4rd]

indeed, this is interesting, haven't seen such issue before, could you print the layout of your project, maybe the tests are run in different package apart from what is being tested and that way it does not detect any coverage on tested package, since go test treats the current package for coverage

[montera82]

this is the directory content of my project ( i.e /go/src/github.com/mycompany/mapper )

├── cmd
│   ├── api
│   └── consumer
├── config
├── domain
├── errors
├── features
│   └── bootstrap
├── infrastructure
└── vendor

some background

My project consists of a rest api, and a consumer which work in this manner :

The api accepts restul http requests and post them in a rabbitmq exchange, the consumer picks up these messages and persists them in a datastore.

./main_test.go ( which contains func main) exists in the root directory (same level as cmd ) and is of package mapper

The contents inside cmd/api and cmd/consumer are package main so that they can be built as separate binaries..both which contain the logic i am testing with godog in features/*

The godog tests in features/* basically make http calls to the endpoints defined in cmd/api. To trigger them, i basically hijack the docker container where the api and consumer are built and runing, and then i type go test -covermode=atomic -coverprofile=cover.txt on the console. And like i mentioned, i see the output of the godog tests runing successfully, but no coverage is recorded.

update

i tried changing the package of main_test.go to package main still the same result.

[l3pp4rd]

The reason why you do not get any coverage, is because you test packages outside the scope of the package you run tests. Even, if you replace godog tests with standard go tests, you will get the same zero coverage.

The test command should be run in the package you test in order to get coverage. since in your step definitions, you include other packages to test their public interface, you get zero coverage, since that package is imported, but is not the current package you start go test in. Do you understand the reason?

In golang everything is based per package, see the issue https://github.com/golang/go/issues/6909 So far I think there are tools, to calculate imported package coverage, but that is not so far a built in go command feature.

[montera82]

hi @l3pp4rd indeed this is happening because my project has its steps definition located in another package i.e in features/bootstrap, and thus were importing the packages in cmd/* also looking more closely in godog docs, i could see that _test.go is appended to the end of the step definition files which i was not doing for my godog step definition files.

Now i'll try to refactor my project accordingly so that i get the coverage metrics from my godog tests. i read how the guys at elastic search did some thing nearly related ( although they used an internal built testing tool) ..

thanks!

[l3pp4rd]

regarding step definitions, it is fine to import it. you do not need to refactor it. since you import step definitions in test.go file it will never be a part of test coverage. the reason why you do not have test coverage is that you import the packages where your actual source code is, which you are testing. If you run go test from inside that package where your source code is then it will be included in coverage.