rogpeppe / go-internal

Selected Go-internal packages factored out from the standard library
BSD 3-Clause "New" or "Revised" License
823 stars 67 forks source link

testscript: generating cover profile from binary compiled with `go build -cover` #250

Open thehowl opened 2 months ago

thehowl commented 2 months ago

We currently have a testscript set up to run our executable using cmd/exec; before running the script proper, our internal code for integration testing ensures to build the executable using -cover, if testing.CoverMode() != "".

We are doing this to ensure to mimick as much as possible a real execution scenario and avoid any problems related to the executable running Setenv/Chdir -- or in any way working with a "dirty" state.

At the time being, we've set up a different directory which is used as the GOCOVERDIR when running the subprocesses. That way, we can then use go tool covdata as outlined in the build-cover blogpost.

// SetupTestscriptsCoverage sets up the given testscripts environment for coverage.
// It will mostly override `GOCOVERDIR` with the target cover directory
func SetupTestscriptsCoverage(p *testscript.Params, coverdir string) error {
    // [elided: create coverdir if it doesn't exist, clean the arg]

    // Backup the original setup function
    origSetup := p.Setup
    p.Setup = func(env *testscript.Env) error {
        if origSetup != nil {
            // Call previous setup first
            origSetup(env)
        }

        // Override `GOCOVEDIR` directory for sub-execution
        env.Setenv("GOCOVERDIR", coverdir)
        return nil
    }

    return nil
}

However, I was trying to figure out if it is in any way possible to simplify this workflow and include the coverage data from the sub-binary directly in the cover profile generated by go test (the one specified in go test -coverprofile).

Inspecting through cmd/go/internal/test, this doesn't seem to be a feature supported "directly": mergeCoverProfile, which performs the merging into the eventual coverage profile, is only called for what seem to be files explicitly created by go test. So, what I'm trying to ask is whether there is any known or suggested workaround to make this work; or if the recommended way forward (provided we still want to build the executable as a separate binary) is to keep two different coverage reports.

Thanks!