cmd/go: go mod download does not report invalid certificate error

nmiyake commented 4 years ago

What version of Go are you using (`go version`)?

go version go1.13.3 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

golang:1.13.3 Docker image:

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build897368373=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Using a network that performs TLS MiTM decryption (that is, uses an internal root certificate for TLS traffic so that network inspection can be performed):

# docker run -it golang:1.13.3
# GOPROXY=direct go mod download -json rsc.io/sampler@v1.3.1
go: finding rsc.io/sampler v1.3.1
{
        "Path": "rsc.io/sampler",
        "Version": "v1.3.1",
        "Error": "rsc.io/sampler@v1.3.1: invalid version: unknown revision v1.3.1"
}

What did you expect to see?

The fetch operation is actually failing because the fetch operation can't validate the certificate injected by the TLS MiTM decryption. Verified this by adding the root certificate to the trust store and running the same command again:

# wget -qO- http://{URL_FOR_CERT} >> /etc/ssl/certs/ca-certificates.crt
# GOPROXY=direct go mod download -json rsc.io/sampler@v1.3.1
go: finding rsc.io/sampler v1.3.1
{
        "Path": "rsc.io/sampler",
        "Version": "v1.3.1",
        "Info": "/go/pkg/mod/cache/download/rsc.io/sampler/@v/v1.3.1.info",
        "GoMod": "/go/pkg/mod/cache/download/rsc.io/sampler/@v/v1.3.1.mod",
        "Zip": "/go/pkg/mod/cache/download/rsc.io/sampler/@v/v1.3.1.zip",
        "Dir": "/go/pkg/mod/rsc.io/sampler@v1.3.1",
        "Sum": "h1:F0c3J2nQCdk9ODsNhU3sElnvPIxM/xV1c/qZuAeZmac=",
        "GoModSum": "h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA="
}

I would expect the original error to state that it could not determine the version because the network call it made could not validate the certificate -- because the error didn't make this clear, it took trial-and-error debugging to determine that the TLS decryption policy was the cause for this failure.

What did you see instead?

Error that did not describe the root cause:

{
        "Path": "rsc.io/sampler",
        "Version": "v1.3.1",
        "Error": "rsc.io/sampler@v1.3.1: invalid version: unknown revision v1.3.1"
}

nmiyake commented 4 years ago

May be related to #30134, although in this case, I feel like a better error message would be more warranted than an specific exit code, since even if it failed with a non-standard exit code I wouldn't have known to check for that to determine that this was an issue with certificate validation.

bcmills commented 4 years ago

@nmiyake, this may have been fixed incidentally along with the fix for #30748. Could you try building from head (or a fresh download using golang.org/dl/gotip) and see whether it is still reproducible?

nmiyake commented 4 years ago

@bcmills just tried using gotip and see the same issue/failure:

# docker run -it golang:1.13.3
# go get golang.org/dl/gotip
# gotip download
# gotip version
go version devel +03978a9 Mon Oct 21 17:47:24 2019 +0000 linux/amd64
# GOPROXY=direct gotip mod download -json rsc.io/sampler@v1.3.1
go: finding rsc.io/sampler v1.3.1
{
    "Path": "rsc.io/sampler",
    "Version": "v1.3.1",
    "Error": "rsc.io/sampler@v1.3.1: invalid version: unknown revision v1.3.1"
}
# wget -qO- http://{URL_FOR_CERT} >> /etc/ssl/certs/ca-certificates.crt
# GOPROXY=direct gotip mod download -json rsc.io/sampler@v1.3.1
go: finding rsc.io/sampler v1.3.1
{
    "Path": "rsc.io/sampler",
    "Version": "v1.3.1",
    "Info": "/go/pkg/mod/cache/download/rsc.io/sampler/@v/v1.3.1.info",
    "GoMod": "/go/pkg/mod/cache/download/rsc.io/sampler/@v/v1.3.1.mod",
    "Zip": "/go/pkg/mod/cache/download/rsc.io/sampler/@v/v1.3.1.zip",
    "Dir": "/go/pkg/mod/rsc.io/sampler@v1.3.1",
    "Sum": "h1:F0c3J2nQCdk9ODsNhU3sElnvPIxM/xV1c/qZuAeZmac=",
    "GoModSum": "h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA="
}

bcmills commented 4 years ago

Thanks for confirming.

gmichelo commented 4 years ago

I tried to reproduce this issue but I cannot. I set up a dummy proxy with untrusted certificate. Here is what I get:

# docker run -it golang:1.13.3
# ./dummyproxy &
# GOPROXY=localhost:4242 go mod download -json rsc.io/sampler@v1.3.1
go: finding rsc.io/sampler v1.3.1
{
    "Path": "rsc.io/sampler",
    "Version": "v1.3.1",
    "Error": "rsc.io/sampler@v1.3.1: Get https://localhost:4242/rsc.io/sampler/@v/v1.3.1.info: x509: certificate signed by unknown authority"
}

TLS MiTM decryption is different from specifying the the GOPROXY explicitly as I did. But at the end of the day go mod download should get the wrong certificate in both ways. Am I missing something?

golang / go