cmd/go: document how to get 'go mod tidy' to keep an unused dependency

[carnott-snap]

What version of Go are you using (go version)?

$ go version
go version go1.13.8 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/user/.cache/go-build"
GOENV="/home/user/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/user/go"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/user/.local/share/umake/go/go-lang"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/user/.local/share/umake/go/go-lang/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/tmp/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build847180785=/tmp/go-build -gno-record-gcc-switches"

What did you do?

In light of the recent golang.org/x/crypto fix, I tried to enforce that my module use v0.0.0-20200220183623-bac4c82f6975:

// /tmp/go.mod
module tmp

go 1.13

require (
        golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975 // indirect
        lib v0.0.0-00010101000000-000000000000
)

replace lib => ./lib

golang.org/x/crypto is listed in my go.sum because a module I depend upon uses it:

// /tmp/go.sum
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550 h1:ObdrDkeb4kJdCP557AjRjq69pTHfNouLtWZG7j9rPN8=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=

// /tmp/main.go
package main

import _ "lib"

func main() {}

It is assumed that I am not in control of the lib module:

// /tmp/lib/go.mod
module lib

go 1.13

require golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550

// /tmp/lib/lib.go
package lib

// /tmp/lib/sub/sub.go
package sub

import _ "golang.org/x/crypto/acme"

However it is not needed for build, as evidenced by go mod why. As such, when I go mod tidy, it gets dropped:

[user@localhost tmp]$ go mod why -m golang.org/x/crypto
# golang.org/x/crypto
(main module does not need module golang.org/x/crypto)
[user@localhost tmp]$ go mod tidy
[user@localhost tmp]$ cat go.mod

What did you expect to see?

// /tmp/go.mod
module test

go 1.13

require (
        golang.org/x/crypto v0.0.0-20200220183623-bac4c82f6975 // indirect
        lib v0.0.0-00010101000000-000000000000
)

replace lib => ./lib

What did you see instead?

// /tmp/go.mod
module test

go 1.13

require lib v0.0.0-00010101000000-000000000000

replace lib => ./lib

[bcmills]

it is not needed for build, as evidenced by go mod why. As such, when I go mod tidy, it gets dropped

That is exactly what go mod tidy is documented to do (https://tip.golang.org/cmd/go/#hdr-Add_missing_and_remove_unused_modules):

it removes unused modules that don't provide any relevant packages.

Could you provide some more detail as to why you want to upgrade a module that does not provide any imported packages?

[carnott-snap]

That is exactly what go mod tidy is documented to do (https://tip.golang.org/cmd/go/#hdr-Add_missing_and_remove_unused_modules):

This is not just in tip; thanks for the pointer.

Could you provide some more detail as to why you want to upgrade a module that does not provide any imported packages?

1. I have tooling that detects vulnerable modules in use from go.sum and go.mod. It flagged my go.sum, as it looks the same as if the module was actually being used for build.
2. Since crypto is a transitive dependency, it would be easy for another developer to introduce a dependency upon sub without realising that they were bringing in the vulnerable version.
   • This is technically true for different modules too, but since we know lib depends on the vulnerable version, I would like a way to flag/upgrade it.

[bcmills]

It flagged my go.sum, as it looks the same as if the module was actually being used for build.

Ah, #33008 could be related. For modules that do not contribute imported packages, the go.sum file is only supposed to contain a checksum for the go.mod file (not the full source code), but a bug in go mod tidy causes it to also retain a checksum for the source code if present.

[bcmills]

Since crypto is a transitive dependency, it would be easy for another developer to introduce a dependency upon sub without realising that they were bringing in the vulnerable version.

That's true, but we should probably provide some means to flag those regardless of how they are introduced. (That's #24031.)

[bcmills]

since we know lib depends on the vulnerable version, I would like a way to flag/upgrade it.

Note that if you really want to, you can introduce a dummy import of some package in order to convince go mod tidy to keep it. (See http://golang.org/wiki/Modules#how-can-i-track-tool-dependencies-for-a-module.)

[bcmills]

Since we have issues filed for the other two parts, let's focus this one on the third: you have a reason to want to retain an otherwise-unused requirement, and we have a mechanism for that (a dummy import), so we should probably make the documentation for that mechanism more discoverable.

[justin-calleja]

Is having a hugo module require but not used in any .go file (because I'm just pulling in content), a valid use case here? I can get it to work but I have to live with the "is not used in this module" warnings in vscode problems tab.

[bcmills]

Hugo's use of modules isn't supported by the Go project — we're focused on the use-case of building and running Go binaries using go build, go test, and the like.

(We're not actively trying to break it either, but you'll need to take up any Hugo-specific issues with the Hugo maintainers.)

[justin-calleja]

@bcmills fair enough. Thanks :)