help wanted on usage of gobinarycoverage

[Arti34]

Hi team,

In the usage part of this library documentation, it is mentioned that Call gobinarycoverage < package-name >. but since this is a go file we can not execute it from the command line. I want to use this library for coverage purpose but do not get how to execute it on the package-name. can you please mention the steps on how you all execute it? @oleorhagen if you could help here.

[oleorhagen]

Hi @Arti34 :wave:

it is mentioned that Call gobinarycoverage < package-name >

First you need to install the binary. So you need to have to installed on your system, then running go build gobinarycoverage.go will give you a binary called gobinarycoverage. To install it on your system, run: go install gobinarycoverage.go.

Then, as an explicit example, I have a service, called deployments, in $HOME/go/src/github.com/mendersoftware/deployments.

If I cd to this directory, and then do:

gobinarycoverage github.com/mendersoftware/deployments, you will see that a lot of your files have now changed.

This is what main.go now looks like in this repo:

package main

import (
    "fmt"
    "io/ioutil"
    "testing"

    _cover0 "github.com/mendersoftware/deployments/api/http"

    _cover1 "github.com/mendersoftware/deployments/app"

    _cover2 "github.com/mendersoftware/deployments/client/inventory"

    _cover3 "github.com/mendersoftware/deployments/client/reporting"

    _cover4 "github.com/mendersoftware/deployments/client/workflows"

    _cover5 "github.com/mendersoftware/deployments/config"

    _cover6 "github.com/mendersoftware/deployments/model"

    _cover7 "github.com/mendersoftware/deployments/s3"

    _cover8 "github.com/mendersoftware/deployments/store"

    _cover9 "github.com/mendersoftware/deployments/store/mongo"

    _cover10 "github.com/mendersoftware/deployments/utils"

    _cover11 "github.com/mendersoftware/deployments/utils/restutil"

    _cover12 "github.com/mendersoftware/deployments/utils/restutil/view"
    "context"
    "crypto/rand"
    "encoding/base64"
    "fmt"
    "io"
    "os"
    "strings"

    "github.com/mendersoftware/go-lib-micro/config"
    "github.com/mendersoftware/go-lib-micro/log"
    mstore "github.com/mendersoftware/go-lib-micro/store"
    "github.com/pkg/errors"
    "github.com/urfave/cli"

    dconfig "github.com/mendersoftware/deployments/config"
    "github.com/mendersoftware/deployments/store/mongo"
)

var (
    coverCounters   = make(map[string][]uint32)
    coverBlocks = make(map[string][]testing.CoverBlock)
)

func init() {

    coverRegisterFile("github.com/mendersoftware/deployments/api/http/api_deployments.go", _cover0.GoCover1.Count[:], _cover0.GoCover1.Pos[:], _cover0.GoCover1.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/api/http/rest_view.go", _cover0.GoCover2.Count[:], _cover0.GoCover2.Pos[:], _cover0.GoCover2.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/api/http/routing.go", _cover0.GoCover3.Count[:], _cover0.GoCover3.Pos[:], _cover0.GoCover3.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/app/app.go", _cover1.GoCover1.Count[:], _cover1.GoCover1.Pos[:], _cover1.GoCover1.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/client/inventory/client.go", _cover2.GoCover1.Count[:], _cover2.GoCover1.Pos[:], _cover2.GoCover1.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/client/reporting/client.go", _cover3.GoCover1.Count[:], _cover3.GoCover1.Pos[:], _cover3.GoCover1.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/client/workflows/client.go", _cover4.GoCover1.Count[:], _cover4.GoCover1.Pos[:], _cover4.GoCover1.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/config/config.go", _cover5.GoCover1.Count[:], _cover5.GoCover1.Pos[:], _cover5.GoCover1.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/model/configuration_deployment.go", _cover6.GoCover1.Count[:], _cover6.GoCover1.Pos[:], _cover6.GoCover1.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/model/deployment.go", _cover6.GoCover2.Count[:], _cover6.GoCover2.Pos[:], _cover6.GoCover2.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/model/deployment_instructions.go", _cover6.GoCover3.Count[:], _cover6.GoCover3.Pos[:], _cover6.GoCover3.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/model/device.go", _cover6.GoCover4.Count[:], _cover6.GoCover4.Pos[:], _cover6.GoCover4.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/model/device_deployment.go", _cover6.GoCover5.Count[:], _cover6.GoCover5.Pos[:], _cover6.GoCover5.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/model/device_deployment_log.go", _cover6.GoCover6.Count[:], _cover6.GoCover6.Pos[:], _cover6.GoCover6.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/model/error.go", _cover6.GoCover7.Count[:], _cover6.GoCover7.Pos[:], _cover6.GoCover7.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/model/filters.go", _cover6.GoCover8.Count[:], _cover6.GoCover8.Pos[:], _cover6.GoCover8.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/model/image.go", _cover6.GoCover9.Count[:], _cover6.GoCover9.Pos[:], _cover6.GoCover9.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/model/limit.go", _cover6.GoCover10.Count[:], _cover6.GoCover10.Pos[:], _cover6.GoCover10.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/model/link.go", _cover6.GoCover11.Count[:], _cover6.GoCover11.Pos[:], _cover6.GoCover11.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/model/newtenant_req.go", _cover6.GoCover12.Count[:], _cover6.GoCover12.Pos[:], _cover6.GoCover12.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/model/release.go", _cover6.GoCover13.Count[:], _cover6.GoCover13.Pos[:], _cover6.GoCover13.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/model/signature.go", _cover6.GoCover14.Count[:], _cover6.GoCover14.Pos[:], _cover6.GoCover14.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/model/status.go", _cover6.GoCover15.Count[:], _cover6.GoCover15.Pos[:], _cover6.GoCover15.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/model/storage_settings.go", _cover6.GoCover16.Count[:], _cover6.GoCover16.Pos[:], _cover6.GoCover16.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/model/update.go", _cover6.GoCover17.Count[:], _cover6.GoCover17.Pos[:], _cover6.GoCover17.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/model/validation.go", _cover6.GoCover18.Count[:], _cover6.GoCover18.Pos[:], _cover6.GoCover18.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/s3/filestorage.go", _cover7.GoCover1.Count[:], _cover7.GoCover1.Pos[:], _cover7.GoCover1.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/store/datastore.go", _cover8.GoCover1.Count[:], _cover8.GoCover1.Pos[:], _cover8.GoCover1.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/store/query.go", _cover8.GoCover2.Count[:], _cover8.GoCover2.Pos[:], _cover8.GoCover2.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/store/mongo/datastore_mongo.go", _cover9.GoCover1.Count[:], _cover9.GoCover1.Pos[:], _cover9.GoCover1.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/store/mongo/migration_1_2_1.go", _cover9.GoCover2.Count[:], _cover9.GoCover2.Pos[:], _cover9.GoCover2.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/store/mongo/migration_1_2_2.go", _cover9.GoCover3.Count[:], _cover9.GoCover3.Pos[:], _cover9.GoCover3.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/store/mongo/migration_1_2_3.go", _cover9.GoCover4.Count[:], _cover9.GoCover4.Pos[:], _cover9.GoCover4.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/store/mongo/migration_1_2_4.go", _cover9.GoCover5.Count[:], _cover9.GoCover5.Pos[:], _cover9.GoCover5.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/store/mongo/migration_1_2_5.go", _cover9.GoCover6.Count[:], _cover9.GoCover6.Pos[:], _cover9.GoCover6.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/store/mongo/migration_1_2_6.go", _cover9.GoCover7.Count[:], _cover9.GoCover7.Pos[:], _cover9.GoCover7.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/store/mongo/migration_1_2_7.go", _cover9.GoCover8.Count[:], _cover9.GoCover8.Pos[:], _cover9.GoCover8.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/store/mongo/migration_1_2_8.go", _cover9.GoCover9.Count[:], _cover9.GoCover9.Pos[:], _cover9.GoCover9.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/store/mongo/migrations.go", _cover9.GoCover10.Count[:], _cover9.GoCover10.Pos[:], _cover9.GoCover10.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/utils/io.go", _cover10.GoCover1.Count[:], _cover10.GoCover1.Pos[:], _cover10.GoCover1.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/utils/restutil/options.go", _cover11.GoCover1.Count[:], _cover11.GoCover1.Pos[:], _cover11.GoCover1.NumStmt[:])

    coverRegisterFile("github.com/mendersoftware/deployments/utils/restutil/view/view.go", _cover12.GoCover1.Count[:], _cover12.GoCover1.Pos[:], _cover12.GoCover1.NumStmt[:])

}

func coverRegisterFile(fileName string, counter []uint32, pos []uint32, numStmts []uint16) {
    if 3*len(counter) != len(pos) || len(counter) != len(numStmts) {
        panic("coverage: mismatched sizes")
    }
    if coverCounters[fileName] != nil {

        return
    }
    coverCounters[fileName] = counter
    block := make([]testing.CoverBlock, len(counter))
    for i := range counter {
        block[i] = testing.CoverBlock{
            Line0:  pos[3*i+0],
            Col0:   uint16(pos[3*i+2]),
            Line1:  pos[3*i+1],
            Col1:   uint16(pos[3*i+2] >> 16),
            Stmts:  numStmts[i],
        }
    }
    coverBlocks[fileName] = block
}

func coverReport() {

    reportFile, err := ioutil.TempFile(os.Getenv("COVERAGE_FILEPATH"), "coverage"+os.Getenv("COVERAGE_FILENAME")+"*.out")
    if err != nil {
        return
    }

    fmt.Fprintf(reportFile, "mode: count\n")

    var active, total int64
    for name, counts := range coverCounters {
        blocks := coverBlocks[name]
        for i := range counts {
            stmts := int64(blocks[i].Stmts)
            total += stmts
            if counts[i] > 0 {
                active += stmts
            }
            fmt.Fprintf(reportFile, "%s:%d.%d,%d.%d %d %d\n", name,
                blocks[i].Line0, blocks[i].Col0,
                blocks[i].Line1, blocks[i].Col1,
                stmts,
                counts[i])
        }
    }
    if total == 0 {
        fmt.Fprintln(os.Stderr, "coverage: [no statements]")
        return
    }
    fmt.Fprintf(os.Stderr, "coverage: %.1f%% of statements %s\n", 100*float64(active)/float64(total), "github.com/mendersoftware/mender")
    fmt.Fprintf(os.Stderr, "Wrote coverage to the file: %s\n", reportFile.Name())

}
func main() {
    doMain(os.Args)
}

func doMain(args []string) {

    var configPath string

    app := cli.NewApp()
    app.Usage = "Deployments Service"

    app.Flags = []cli.Flag{
        cli.StringFlag{
            Name:   "config",
            Usage: "Configuration `FILE`." +
                " Supports JSON, TOML, YAML and HCL formatted configs.",
            Destination:    &configPath,
        },
    }

    app.Commands = []cli.Command{
        {
            Name:   "server",
            Usage:  "Run the service as a server",
            Flags: []cli.Flag{
                cli.BoolFlag{
                    Name:   "automigrate",
                    Usage:  "Run database migrations before starting.",
                },
            },

            Action: cmdServer,
        },
        {
            Name:   "migrate",
            Usage:  "Run migrations and exit",
            Flags: []cli.Flag{
                cli.StringFlag{
                    Name:   "tenant",
                    Usage:  "Tenant ID (optional).",
                },
            },

            Action: cmdMigrate,
        },
    }

    app.Action = cmdServer
    app.Before = func(args *cli.Context) error {

        l := log.NewEmpty()
        err := config.FromConfigFile(configPath, dconfig.Defaults)
        if err != nil {
            return cli.NewExitError(
                fmt.Sprintf("error loading configuration: %s", err),
                1)
        }

        config.Config.SetEnvPrefix("DEPLOYMENTS")
        config.Config.AutomaticEnv()
        config.Config.SetEnvKeyReplacer(strings.NewReplacer(".", "_", "-", "_"))
        if config.Config.Get(dconfig.SettingPresignSecret) == "" {
            l.Infof("'%s' not configured. Generating a random secret.",
                dconfig.SettingPresignSecret,
            )
            var buf [32]byte
            n, err := io.ReadFull(rand.Reader, buf[:])
            if err != nil {
                return errors.Wrapf(err,
                    "failed to generate '%s'",
                    dconfig.SettingPresignSecret,
                )
            } else if n == 0 {
                return errors.Errorf(
                    "failed to generate '%s'",
                    dconfig.SettingPresignSecret,
                )
            }
            secret := base64.StdEncoding.EncodeToString(buf[:n])
            config.Config.Set(dconfig.SettingPresignSecret, secret)
        }

        return nil
    }

    err := app.Run(args)
    if err != nil {
        log.NewEmpty().Fatal(err.Error())
    }
}

func cmdServer(args *cli.Context) error {
    devSetup := args.GlobalBool("dev")

    l := log.New(log.Ctx{})

    if devSetup {
        l.Infof("setting up development configuration")
        config.Config.Set(dconfig.SettingMiddleware, dconfig.EnvDev)
    }

    l.Print("Deployments Service starting up")
    err := migrate("", args.Bool("automigrate"))
    if err != nil {
        return err
    }

    err = RunServer(config.Config)
    if err != nil {
        return cli.NewExitError(err.Error(), 4)
    }

    return nil
}

func cmdMigrate(args *cli.Context) error {
    tenant := args.String("tenant")
    return migrate(tenant, true)
}

func migrate(tenant string, automigrate bool) error {
    ctx := context.Background()

    dbClient, err := mongo.NewMongoClient(ctx, config.Config)
    if err != nil {
        return cli.NewExitError(
            fmt.Sprintf("failed to connect to db: %v", err),
            3)
    }
    defer func() {
        _ = dbClient.Disconnect(ctx)
    }()

    if tenant != "" {
        db := mstore.DbNameForTenant(tenant, mongo.DbName)
        err = mongo.MigrateSingle(ctx, db, mongo.DbVersion, dbClient, true)
    } else {
        err = mongo.Migrate(ctx, mongo.DbVersion, dbClient, true)
    }
    if err != nil {
        return cli.NewExitError(
            fmt.Sprintf("failed to run migrations: %v", err),
            3)
    }

    return nil
}

So everything is now basically ready for collecting coverage. Except that coverReport() is never called.

So this will have to be manually added to main before the binary exits in main. So main should now look similar to:

func main() {
    ret = doMain()
    coverReport()
    os.Exit(ret)
}

Does this make sense?

If it does, I would very much appreciate a PR to help with the README here :smile:

[Arti34]

Thanks, @oleorhagen , before I could see your reply, I directly tried to install it this way: go install github.com/mendersoftware/gobinarycoverage@latest . after that, I was able to execute- Call gobinarycoverage < package-name >.

==================================

but then when I followed the steps you have mentioned above:

It gave me 2 errors: Failed to parse the file: C:...\controllers/main.go. Error: open C:...\controllers/main.go: The system cannot find the file specified. Failed to parse main.go

I thought this might be because my main.go resides in some other package. Since we follow the different package structure so different files reside in a different package.

but to make it work at least for the controllers package, I have added the new main.go in it(which is exact replica of my original main.go)

Then I again tried to execute the gobinarycoverage < package-name >. This time I can see the changes in the newly added main.go, but in that file, I can see only half of the things compared to yours: I can see below chnages that were newly added:

var (
coverCounters = make(map[string][]uint32) coverBlocks = make(map[string][]testing.CoverBlock) )
func init() {
}
func coverRegisterFile(fileName string, counter []uint32, pos []uint32, numStmts []uint16) {
if 3len(counter) != len(pos) || len(counter) != len(numStmts) {
panic("coverage: mismatched sizes") }
if coverCounters[fileName] != nil { return
}
coverCounters[fileName] = counter
block := make([]testing.CoverBlock, len(counter))
for i := range counter {
block[i] = testing.CoverBlock{
Line0: pos[3i+0], Col0: uint16(pos[3i+2]), Line1: pos[3i+1], Col1: uint16(pos[3i+2] >> 16),
Stmts: numStmts[i],
}
}
coverBlocks[fileName] = block
}
func coverReport() {
reportFile, err := ioutil.TempFile(os.Getenv("COVERAGE_FILEPATH"), "coverage"+os.Getenv("COVERAGE_FILENAME")+".out")
if err != nil { return
}
fmt.Fprintf(reportFile, "mode: count\n")
var active, total int64 for name, counts := range coverCounters {
blocks := coverBlocks[name] for i := range counts { stmts := int64(blocks[i].Stmts) total += stmts
if counts[i] > 0 {
active += stmts }
fmt.Fprintf(reportFile, "%s:%d.%d,%d.%d %d %d\n", name, blocks[i].Line0, blocks[i].Col0,
blocks[i].Line1, blocks[i].Col1,
stmts,
counts[i])
}
}
if total == 0 { fmt.Fprintln(os.Stderr, "coverage: [no statements]")
return
}
fmt.Fprintf(os.Stderr, "coverage: %.1f%% of statements %s\n", 100*float64(active)/float64(total), "github.com/mendersoftware/mender")
fmt.Fprintf(os.Stderr, "Wrote coverage to the file: %s\n", reportFile.Name())
}

==================================

Apart from this in import only this were added: "io/ioutil" "testing"

there were no lines starting with _cover were added and the init function is also empty.

==================================

• Is it compulsory that the main.go and the rest of the go files should be in the same package/directory?
• If I want it to make it work for our packaging structure wherein we have folders within folders and main.go resides in some other folder, what change do I need to make? so that it will work in one go.

[oleorhagen]

@Arti34 is this project open source?

If so I can have a look at what is going on, and help you on the way here?

[Arti34]

@oleorhagen, No it's not open source, but the structure is something like the below example structure: In the above, you can see that in service there is again another folder which again has go files and some service files in parallel.

Any specific thing you wanted to check maybe if I can confirm that and let you know.

[oleorhagen]

Hmm, I see.

I do think it should be enough to call gobinarycoverage <package name of main.go file>.

There should be no need of adding extra main.go files.

The tools checks the imports from the main.go file, and goes through these packages as well, and instruments them for coverage. You can also specify multiple packages I think.

So try: gobinarycoverage <main.go package> <other package 1> <other package 2> and see how you fare?

Also, I am curious, how did you find this project? What was your need?

[Arti34]

Okay, I will try this gobinarycoverage <other package 1> <other package 2> and let you know.

I want to capture my service's code coverage with external tests e.g. by running robot tests

[oleorhagen]

@Arti34 how did it go?

[oleorhagen]

Thinking no news is good news here. Closing