cmd/test2json: CGO_ENABLED is set, when running tests from prebuilt binary

cardil commented 2 years ago

What version of Go are you using (`go version`)?

$ go version
go version go1.18.1 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ksuszyns/.cache/go-build"
GOENV="/home/ksuszyns/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/ksuszyns/.gvm/pkgsets/go1.18.1/global/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/ksuszyns/.gvm/pkgsets/go1.18.1/global"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/ksuszyns/git/golang-go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/ksuszyns/git/golang-go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.18.1"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2603697588=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Checkout the reproduction repo: https://github.com/cardil/repro-cgo-drift
Run make repro

What did you expect to see?

I expect to see the test pass regardless if executed via:

go test -count=1 ./

or

go test -c -o ./repro.test ./
go tool test2json -t ./repro.test -test.count=1

What did you see instead?

The process fails with the following output:

go test -c -o ./repro.test ./
go tool test2json -t ./repro.test -test.count=1
{"Time":"2022-08-19T12:42:13.742590288+02:00","Action":"output","Test":"TestCgo","Output":"--- FAIL: TestCgo (0.00s)\n"}
{"Time":"2022-08-19T12:42:13.743167948+02:00","Action":"output","Test":"TestCgo","Output":"    cgo_test.go:12: Expecting CGO_ENABLED equal \"\", got \"1\"\n"}
{"Time":"2022-08-19T12:42:13.743197548+02:00","Action":"fail","Test":"TestCgo","Elapsed":0}
{"Time":"2022-08-19T12:42:13.743212794+02:00","Action":"output","Output":"FAIL\n"}
{"Time":"2022-08-19T12:42:13.743499898+02:00","Action":"fail","Elapsed":0.006}

Additional context

I stumbled on this problem while running tests via the Goland IDE. The Goland IDE uses the pair of commands to execute tests:

go test -c -o /tmp/Goland/test.bin ./package/path
go tool test2json -t /tmp/Goland/test.bin -test.v -test.paniconexit0

We are executing google/ko in our tests to built test OCI images. KO sets the CGO_ENABLED=0 by default because the default base image is distroless, which doesn't contain any system libraries.

When I execute tests from Goland IDE, I have effectively CGO_ENABLED=1 when the test starts. This overrides the KO default value, and the produced image is invalid. The faulty image contains dynamic binary, which can't run on distroless.

When the tests are executed with go test directly, all is well.

I think running tests via go tool test2json -t shouldn't change the execution environment for the test.

cardil commented 2 years ago

I think the following fragment is responsible for this discrepancy:

https://github.com/golang/go/blob/6001c043dc067b7f6b9a2e934439b6a7948f92ef/src/cmd/go/main.go#L202-L213

seankhliao commented 2 years ago

Arguably the test is both fragile and wrong, it tests for an unset value (ie use the default) when it actually needs the specific value of off.

gopherbot commented 2 years ago

Change https://go.dev/cl/424999 mentions this issue: cmd/test2json: use original environment to mirror go test

cardil commented 2 years ago

The point is that the test2json uses effective env - same as go env, but go test actually uses the original env: https://github.com/golang/go/blob/bf4e35b658e61c29112c456f47615c16345c3518/src/cmd/go/internal/test/test.go#L1358

The PR #54554 align this to be the same.

bcmills commented 2 years ago

go tool test2json is not a supported way to run tests. If you want precise control over the process's environment, use go tool -n test2json to obtain the path to the tool, and then run it with whatever environment you need for it.

golang / go