cmd/go: restore ability to see what updates will happen

[thepudds]

In 1.12, go list -u -m all displays the direct and indirect dependencies of the current module that have available updates, and go get -u attempts to update those dependencies.

In #26902, there was a very nice change that narrowed the set of updates that happen for things like go get -u -m all (and other variations like go get -u all), which now no longer follow the full module-level dependency graph.

However, now it is not easy to see what those updates will be:

• go list -u -m all still shows information from the module-level dependency graph.
• updates via go get -u -m all or similar pay attention to the package-level import graph to determine what updates actually happen, which is often a subset of what is shown by go list -u -m all.

It would be nice if something could be done to make the "what updates are available" and "please do those updates" workflow more consistent and easy to remember.

Comments from CL 174099 discussion

Moving over some comments from https://golang.org/cl/174099:

It would be nice if there was some 'go list' invocation that could show the list of available upgrades under the new 'go get' behavior. One could try to do something today on tip by chaining together different commands, such as something like:

 $ go list -u -m $(go list -deps -test -f '{{with .Module}}{{.Path}}{{end}}')

That might be close, but I am not sure if that is correct, and clearly that is not convenient.

Putting the above comments together-- it would be nice if there was a recommended way to do something like:

 1. go list <some args>
 2. go get <same args>
 3. go list <same args>

and have step 3. report no more upgrades are available. It would be nice if the same exact arguments could be used in each of those steps (although having them be the same arguments might not end up being possible or desirable based on other constraints).

For example, perhaps 'go list -u -m' could be defined in 1.13 to show the upgrades available under the new 'go get -u -m' behavior. That would mean you could do:

 1. go list -u -m       // upgrades listed
 2. go get -u -m        // upgrades happen
 3. go list -u -m       // no upgrades listed

Alternatively, perhaps the meaning of 'all' for 'go list' could be made to conform to the newly narrower meaning meaning of 'all' for 'go get', and then you could do:

 1. go list -u -m all   // upgrades listed
 2. go get -u -m all    // upgrades happen
 3. go list -u -m all   // no upgrades listed

Or perhaps there could be some other change to smooth things out here.

Jay responded in CL 174099:

I don't think there's a concise command for "check what modules go get -u all" would upgrade. 'go list -m -u all' will list modules that are part of the module graph, but this is a superset of what 'go get -u all' does now. We should probably add a flag to 'go get' that prints what it would do without actually doing it (or maybe we should expand the interpretation of -n). Whatever we do should be consistent with #27005 (same but for 'go mod tidy'). Feel free to open an issue or comment on that one.

Additional Background

The exact behavior for go get might be tweaked further, but in CL 176902, the go get help currently includes these related excerpts describing the new behavior:

 The -u flag instructs get to update modules providing dependencies
 of packages named on the command line to use newer minor or patch
 releases when available. 

 [...]

 The special  pattern 'all' has the same meaning whether or not the -m flag is used:
 'all' matches packages in the main module and their dependencies,
 including test dependencies.

 When the -m and -u flags are used together, 'go get' will update
 modules providing packages imported by packages in the modules named on
 the command line. For example, 'go get -m -u A' will update A and
 any module providing packages needed to build packages in A,
 not including tests. 'go get -m -u' will update modules providing packages
 needed to build packages in the main module. 'go get -u all' also updates
 modules providing packages needed to build packages in the main module,
 and it also updates modules providing test dependencies.

Potentially related: #32037, #32038 CC @bcmills, @jayconrod

[bcmills]

The “show what would happen” flag is usually -n, but there is no such flag for go get -u (even in GOPATH mode) today.

It might be reasonable to make go list -u work without -m (it is an error today), in which case you could do: go list -u -f '{{with .Module}}{{with .Update}}{{.Path}} {{.Version}}{{end}}{{end}}' all | sort -u

(Perhaps that's what go list -u should do by default?)

[bcmills]

On the other hand, I would guess that most Go users are using version control anyway: to some extent, the right approach may be to simply run go get -u and look at the resulting diff.

[thepudds]

Given the freeze, it seems reasonable to defer this particular issue until 1.14, but it might be useful to make at least a preliminary decision on what the behavior here would be, especially as possibly related go get tweaks are currently still landing in 1.13 (e.g., #32037, maybe #32038).

(Perhaps that's what go list -u should do by default?)

If go list -u ends up as the future way to see available updates based on the package-level import graph, ideally go get -u would get those updates.

Also, if -t is used for go get -u (#32037), then would it be -test or -t for go list -u, or something else?

Separately, today in 1.12, go list -u -m all is usually faster than go get -u if you need to download the dependencies needed for -u. If the approach taken here ends up being go get -n ... to show what would happen based on a dry run, would that be substantially slower than the go list -u -m all of today? Of course, GOPROXY enabled by default would help with future performance.

[jayconrod]

The “show what would happen” flag is usually -n, but there is no such flag for go get -u (even in GOPATH mode) today.

To complicate matters, go get already accepts -n, but it shows the build steps, not the module operations. So if we added a flag to go get, it would need to be called something else.

I'd prefer a go get flag to a go list flag though. go list won't be able to predict what go get will do, since as we update, packages named on the command line may be provided by different modules and may have new dependencies. The only accurate way to do this is to do everything go get does except actually write out the go.mod file.

Also, if -t is used for go get -u (#32037), then would it be -test or -t for go list -u, or something else?

It's unfortunate that these don't have the same name. go get had a -t flag in GOPATH mode, so I thought it was more important to preserve that. go get -t and go list -test set the same package loading bit though.

Separately, today in 1.12, go list -u -m all is usually faster than go get -u if you need to download the dependencies needed for -u. If the approach taken here ends up being go get -n ... to show what would happen based on a dry run, would that be substantially slower than the go list -u -m all of today? Of course, GOPROXY enabled by default would help with future performance.

When GOPROXY is enabled, most go get operations will be slower in 1.13 than 1.12. In 1.12, we only considered go.mod files, and they can be downloaded from GOPROXY without the rest of the module content. In 1.13, we will load packages specified on the command line, and that requires downloading zips. Without GOPROXY enabled, we have to do a shallow clone either way, so it may not make a huge difference.

In any case, the new behavior is much closer to the GOPATH behavior, so I hope it won't be unacceptably slow.