search

golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
123.33k stars 17.58k forks source link

cmd/go: go install downloads too much #45004

Open aarondl opened 3 years ago

aarondl commented 3 years ago

What version of Go are you using (go version)?

$ go version
go version go1.16.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/aaron/.cache/go-build"
GOENV="/home/aaron/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/aaron/TESTDOWNLOAD/GOPATH/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/aaron/TESTDOWNLOAD/GOPATH"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.2"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2663290531=/tmp/go-build -gno-record-gcc-switches"

What did you do?

mkdir -p ~/TESTDOWNLOAD/GOPATH
cd ~/TESTDOWNLOAD
export GOPATH=$PWD/GOPATH

go install github.com/volatiletech/sqlboiler/v4@latest

What did you expect to see?

I expected to see only sqlboiler/v4 downloaded installed.

What did you see instead?

I additionally saw it downloading source for versions I was not interested in at all nor appear to be part of the dependency tree. Note the extraneous downloads of the source for v1.0.0 and v3.7.1+incompatible.

# get https://proxy.golang.org/github.com/@v/list
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/list
# get https://proxy.golang.org/github.com/volatiletech/@v/list
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/list
# get https://proxy.golang.org/github.com/volatiletech/@v/list: 410 Gone (0.113s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/list: 200 OK (0.118s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/v4.5.0.info
# get https://proxy.golang.org/github.com/@v/list: 410 Gone (0.119s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/list: 200 OK (0.125s)
go: downloading github.com/volatiletech/sqlboiler v1.0.0
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v1.0.0.zip
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/v4.5.0.info: 200 OK (0.009s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/v4.5.0.mod
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v1.0.0.zip: 200 OK (0.009s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/v4.5.0.mod: 200 OK (0.025s)
# get https://proxy.golang.org/sumdb/sum.golang.org/supported
# get https://proxy.golang.org/sumdb/sum.golang.org/supported: 410 Gone (0.008s)
# get https://sum.golang.org/lookup/github.com/volatiletech/sqlboiler/v4@v4.5.0
# get https://sum.golang.org/lookup/github.com/volatiletech/sqlboiler@v1.0.0
# get https://sum.golang.org/lookup/github.com/volatiletech/sqlboiler/v4@v4.5.0: 200 OK (0.095s)
# get https://sum.golang.org/lookup/github.com/volatiletech/sqlboiler@v1.0.0: 200 OK (0.095s)
# get https://sum.golang.org/tile/8/0/x012/871.p/183
# get https://sum.golang.org/tile/8/2/000.p/50
# get https://sum.golang.org/tile/8/1/050.p/71
# get https://sum.golang.org/tile/8/2/000.p/50: 200 OK (0.008s)
# get https://sum.golang.org/tile/8/1/050.p/71: 200 OK (0.009s)
# get https://sum.golang.org/tile/8/0/x012/871.p/183: 200 OK (0.009s)
# get https://sum.golang.org/tile/8/0/430
# get https://sum.golang.org/tile/8/1/001
# get https://sum.golang.org/tile/8/1/001: 200 OK (0.009s)
# get https://sum.golang.org/tile/8/0/430: 200 OK (0.010s)
go: downloading github.com/volatiletech/sqlboiler/v4 v4.5.0
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/v4.5.0.zip
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/v4/@v/v4.5.0.zip: 200 OK (0.011s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v3.7.1+incompatible.info
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v3.7.1+incompatible.info: 200 OK (0.046s)
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v3.7.1+incompatible.mod
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v3.7.1+incompatible.mod: 200 OK (0.008s)
# get https://sum.golang.org/lookup/github.com/volatiletech/sqlboiler@v3.7.1+incompatible
# get https://sum.golang.org/lookup/github.com/volatiletech/sqlboiler@v3.7.1+incompatible: 200 OK (0.008s)
# get https://sum.golang.org/tile/8/0/x004/453
# get https://sum.golang.org/tile/8/1/017
# get https://sum.golang.org/tile/8/0/x004/453: 200 OK (0.008s)
# get https://sum.golang.org/tile/8/1/017: 200 OK (0.009s)
go: downloading github.com/volatiletech/sqlboiler v3.7.1+incompatible
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v3.7.1+incompatible.zip
# get https://proxy.golang.org/github.com/volatiletech/sqlboiler/@v/v3.7.1+incompatible.zip: 200 OK (0.011s)

Inside the project in question, I tried some queries to see if maybe there was something pulling in the old versions:

$ go mod graph | grep sqlboiler
github.com/volatiletech/sqlboiler/v4 github.com/DATA-DOG/go-sqlmock@v1.4.1
github.com/volatiletech/sqlboiler/v4 github.com/davecgh/go-spew@v1.1.1
github.com/volatiletech/sqlboiler/v4 github.com/denisenkom/go-mssqldb@v0.0.0-20200206145737-bbfc9a55622e
github.com/volatiletech/sqlboiler/v4 github.com/ericlagergren/decimal@v0.0.0-20181231230500-73749d4874d5
github.com/volatiletech/sqlboiler/v4 github.com/friendsofgo/errors@v0.9.2
github.com/volatiletech/sqlboiler/v4 github.com/go-sql-driver/mysql@v1.5.0
github.com/volatiletech/sqlboiler/v4 github.com/kat-co/vala@v0.0.0-20170210184112-42e1d8b61f12
github.com/volatiletech/sqlboiler/v4 github.com/lib/pq@v1.2.1-0.20191011153232-f91d3411e481
github.com/volatiletech/sqlboiler/v4 github.com/spf13/cast@v1.3.1
github.com/volatiletech/sqlboiler/v4 github.com/spf13/cobra@v1.0.0
github.com/volatiletech/sqlboiler/v4 github.com/spf13/viper@v1.6.3
github.com/volatiletech/sqlboiler/v4 github.com/volatiletech/null/v8@v8.1.2
github.com/volatiletech/sqlboiler/v4 github.com/volatiletech/randomize@v0.0.1
github.com/volatiletech/sqlboiler/v4 github.com/volatiletech/strmangle@v0.0.1
github.com/volatiletech/sqlboiler/v4 gopkg.in/yaml.v2@v2.2.4

$ go list -m all | grep sqlboiler
github.com/volatiletech/sqlboiler/v4

$ go list all | grep sqlboiler
github.com/volatiletech/sqlboiler/v4
github.com/volatiletech/sqlboiler/v4/boil
github.com/volatiletech/sqlboiler/v4/boilingcore
github.com/volatiletech/sqlboiler/v4/drivers
github.com/volatiletech/sqlboiler/v4/drivers/mocks
github.com/volatiletech/sqlboiler/v4/drivers/sqlboiler-mssql
github.com/volatiletech/sqlboiler/v4/drivers/sqlboiler-mssql/driver
github.com/volatiletech/sqlboiler/v4/drivers/sqlboiler-mysql
github.com/volatiletech/sqlboiler/v4/drivers/sqlboiler-mysql/driver
github.com/volatiletech/sqlboiler/v4/drivers/sqlboiler-psql
github.com/volatiletech/sqlboiler/v4/drivers/sqlboiler-psql/driver
github.com/volatiletech/sqlboiler/v4/importers
github.com/volatiletech/sqlboiler/v4/models
github.com/volatiletech/sqlboiler/v4/queries
github.com/volatiletech/sqlboiler/v4/queries/qm
github.com/volatiletech/sqlboiler/v4/queries/qmhelper
github.com/volatiletech/sqlboiler/v4/templatebin
github.com/volatiletech/sqlboiler/v4/types
github.com/volatiletech/sqlboiler/v4/types/pgeo

Also as a completely separate nit trying go list all without changes in the repo produces the error:

missing go.sum entry for module providing package gopkg.in/check.v1 (imported by gopkg.in/yaml.v2); to add:
    go get -t gopkg.in/yaml.v2@v2.2.4

Running this command changes nothing and the error continues, the fix is in fact running go mod download which adds many lines to the go.sum file. Perhaps I should add these and commit them as the maintainer of the package.

zikaeroh commented 3 years ago

RE: your last point, apparently it's expected that go mod tidy and go mod download do not result in the same go.sum anymore. See #43994 (and the many linked issues and workarounds people have been applying in other repos...).

cherrymui commented 3 years ago

cc @bcmills @jayconrod @matloob

bcmills commented 3 years ago

Those source downloads are because the go command is looking for a package named at ./v4 within module github.com/volatiletech/sqlboiler. For go get that's maybe not such a big deal because the main module will in general already require the module at the correct version, but for go install I agree that it seems excessive.

bcmills commented 3 years ago

Also as a completely separate nit trying go list all without changes in the repo produces the error:

missing go.sum entry for module providing package gopkg.in/check.v1 (imported by gopkg.in/yaml.v2); to add:
  go get -t gopkg.in/yaml.v2@v2.2.4

Running this command changes nothing and the error continues

I'm not able to reproduce that behavior:

sqlboiler$ go list all >/dev/null
missing go.sum entry for module providing package gopkg.in/check.v1 (imported by gopkg.in/yaml.v2); to add:
        go get -t gopkg.in/yaml.v2@v2.2.4

sqlboiler$ go get -t gopkg.in/yaml.v2@v2.2.4

sqlboiler$ go list all >/dev/null

sqlboiler$

However, that sounds like it could be #44749. The fix for that (#44812) just barely missed 1.16.2.