Checksum error, caused SECURITY ERROR

[1939323749]

I'm trying to add this library to my project by go get -u github.com/fullstorydev/grpcurl but got:

go: downloading github.com/fullstorydev/grpcurl v1.8.8
go: github.com/fullstorydev/grpcurl@v1.8.8: verifying module: checksum mismatch
        downloaded: h1:7HPvLgJfwbf93P5eYQGC1t/puN74bspOz4TPYFXHijk=
        sum.golang.org: h1:74MrTXbTlsNEAAhbwc4r2F5P4Qu7Rkyn9BflEer8vss=

SECURITY ERROR
This download does NOT match the one reported by the checksum server.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.

For more information, see 'go help module-auth'.

[jhump]

@1939323749, @PatrLind, I just ran go get github.com/fullstorydev/grpcurl@v1.8.8 in a clean environment, and it worked fine. Is it possible that you have a corrupted module cache? This actually happened to me recently with a different package, where a module in my local module cache was bad and causing the go tool to compute the wrong checksum for the downloaded version of the module.

Sadly, I haven't yet figured out the right incantation to remove just one module from the local cache. But you can clear it out entirely with go clean -modcache. This means that the next few go install ... commands will take much longer, to re-download packages to re-build the module cache. But it will force a re-download of grpcurl, which may fix the error.

If that doesn't fix it, I suspect it could be related to your platform and/or version of Go. So if the above doesn't work, mind pasting the output of go env?

[1939323749]

@jhump Thank you for your reply, but unfortunately, go clean -modcache did not work. I remembered that I previously installed grpcurl1.8.8 using homebrew, so I used brew uninstall grpucurl to remove it. Then I ran brew clearn and go clean -modcache, and finally ran go get -u github.com/fullstorydev/grpcurl, but the result did not change. Here is my go env, which was also installed through homebrew.

go env
GO111MODULE='on'
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/mba/Library/Caches/go-build'
GOENV='/Users/mba/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/mba/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/mba/go'
GOPRIVATE=''
GOPROXY='https://goproxy.io,direct'
GOROOT='/opt/homebrew/Cellar/go/1.21.2/libexec'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/opt/homebrew/Cellar/go/1.21.2/libexec/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.21.2'
GCCGO='gccgo'
AR='ar'
CC='cc'
CXX='c++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/6l/0rnqwtgd2lqfvr4vcry4xjrc0000gn/T/go-build28476534=/tmp/go-build -gno-record-gcc-switches -fno-common'

If there is no solution, I can use it by disabling the checksum. I believe in the quality of your code! :)

[PatrLind]

I have done some more research, and it seems this only happens with the latest version (v1.8.8), and it also only seems to happen when a proxy is used. I think a proxy is always used by default? Sample output:

$ GOPROXY='https://proxy.golang.org,direct' go get github.com/fullstorydev/grpcurl@v1.8.8
go: downloading github.com/fullstorydev/grpcurl v1.8.8
verifying github.com/fullstorydev/grpcurl@v1.8.8/go.mod: checksum mismatch
        downloaded: h1:TRM21TqPbPzHkA9DqSh94oI2g1pD2AFRhLhmGrSht+Q=
        go.sum:     h1:KK04P+qKmc3/fcV54mxtsSKcLLN1lZA/cLIXHpJpRa0=

SECURITY ERROR
This download does NOT match an earlier download recorded in go.sum.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.

For more information, see 'go help module-auth'.

$ GOPROXY='direct' go get github.com/fullstorydev/grpcurl@v1.8.8
go: added github.com/bufbuild/protocompile v0.6.0
go: upgraded github.com/fullstorydev/grpcurl v1.8.7 => v1.8.8
go: upgraded github.com/jhump/protoreflect v1.12.0 => v1.15.2
go: upgraded google.golang.org/genproto v0.0.0-20230525234025-438c736192d0 => v0.0.0-20230526161137-0005af68ea54
go: upgraded google.golang.org/grpc v1.56.0 => v1.57.0

So maybe the release tag v1.8.8 was created and then removed only to be replaced with another v1.8.8, but by then the proxies already had the old copy?

[jhump]

@1939323749, the one thing that looks fishy in your go env output is the GOPROXY value: https://goproxy.io. You might try unsetting your GOPROXY env var to see if that helps.

So maybe the release tag v1.8.8 was created and then removed only to be replaced with another v1.8.8, but by then the proxies already had the old copy?

I don't think that happened since it works fine on my machine (and I did not create the release, so even if that double-tagging happened, my machine should be as suitable as any other to try this out). Although maybe @dragonsinth or @gpassini can chime in if the double-tagging possibly happened by accident. However, I really think I would see the same errors if that were case...

I was hoping this was just related to goproxy.io. FWIW, I am also using proxy.golang.org and it works fine on my machine :/

> export GOPROXY=https://proxy.golang.org
> go clean -modcache
> rm -rf $GOMODCACHE/../sumdb
> go install github.com/fullstorydev/grpcurl/cmd/grpcurl@v1.8.8
go: downloading github.com/fullstorydev/grpcurl v1.8.8
go: downloading google.golang.org/grpc v1.57.0
go: downloading github.com/jhump/protoreflect v1.15.2
go: downloading google.golang.org/protobuf v1.31.0
go: downloading github.com/golang/protobuf v1.5.3
go: downloading github.com/bufbuild/protocompile v0.6.0
go: downloading golang.org/x/net v0.9.0
go: downloading google.golang.org/genproto/googleapis/rpc v0.0.0-20230525234030-28d5490b6b19
go: downloading github.com/envoyproxy/go-control-plane v0.11.1-0.20230524094728-9239064ad72f
go: downloading golang.org/x/sync v0.3.0
go: downloading google.golang.org/genproto v0.0.0-20230526161137-0005af68ea54
go: downloading github.com/cncf/xds/go v0.0.0-20230607035331-e9ce68804cb4
go: downloading github.com/cespare/xxhash/v2 v2.2.0
go: downloading github.com/cncf/udpa/go v0.0.0-20220112060539-c52dc94e7fbe
go: downloading github.com/envoyproxy/protoc-gen-validate v0.10.1
go: downloading google.golang.org/genproto/googleapis/api v0.0.0-20230525234035-dd9d682886f9
go: downloading github.com/cespare/xxhash v1.1.0
go: downloading golang.org/x/text v0.9.0
go: downloading golang.org/x/oauth2 v0.7.0
go: downloading github.com/census-instrumentation/opencensus-proto v0.4.1
go: downloading cloud.google.com/go/compute/metadata v0.2.3
go: downloading cloud.google.com/go/compute v1.19.1
go: downloading cloud.google.com/go v0.110.0

The only other thing I can see different between my environment and the above go env output is that I am using Go 1.20.1, but that env shows Go 1.21.2. I can try installing that newer version to see if that is somehow the culprit. The changelogs for 1.21.1 and 1.21.2 both indicate security fixes in cmd/go, so maybe it's possible? (Though I can't find any bug reports of other users running into this yet.)

@PatrLind, what version of Go are you using? Or maybe you could provide the output of go env in your environment as well?

[jhump]

If there is no solution, I can use it by disabling the checksum. I believe in the quality of your code! :)

I would not recommend that. Even if you think we can be trusted, there could be other actors in your supply chain that cannot be. I think it's worth figuring out what's going on here.

[PatrLind]

Ok, I hadn't tried the go clean -modcache command when I made my comment earlier. After I ran that command it now works and I don't get any security errors. I assume that when I downloaded that module from proxy.golang.org they had the "wrong" version somehow. Me and OP had the error almost the same time, because he/she created the Issue within seconds of me finding it... So the error seems to be resolved, but it clearly happened and the proxy had differing data from this repository for some reason.

[1939323749]

@jhump After I set export GOPROXY=https://proxy.golang.org, it seems that the problem has been resolved. I successfully added this library and then ran go clean -modcache and switched back to the original GOPROXY. The same error occurred again, and I think it is because I did not use the GOSUMDB provided by my GOPROXY. After consulting the documentation, I ran export GOSUMDB=gosum.io+ce6e7565+AY5qEHUk/qmHc5btzW45JVoENfazw8LielDsaI+lEbq6, but the error still occurred. Then I switched to another Go proxy: https://goproxy.cn/, everything works fine. I think there might be some errors with goproxy.io, which caused this library to be corrupted when synchronized to their CDN(I did not encounter any issues when adding other libraries.). This minor issue should be resolved in the next release.

I would not recommend that. Even if you think we can be trusted, there could be other actors in your supply chain that cannot be. I think it's worth figuring out what's going on here.

Thank you for your suggestion. I am trying out this new feature on my toy project and will need to carefully consider it before putting it into production.