Proposal: improve UX for major module versions

[adg]

DEPRECATED

Please see the new proposals #40357 and #40323 that supersede this one.

---

Proposal: improve UX for major module versions

Peter Bourgon (@peterbourgon), Andrew Gerrand (@adg)

Problem statement

When a user wants to use a module, it is the v0/v1 version of that module which is most prominent, as it is selected by the base repository path: github.com/user/repo is in effect a constraint to v0.x.y/v1.x.y.

To use v2 or above, Semantic Import Versioning requires that the major version number is a suffix of the module path: github.com/user/repo/v2 (constrained to v2.x.y), github.com/user/repo/v3 (constrained to v3.x.y), and so on.

It’s easy for module consumers to default to v0/v1, even if that version is obsoleted by a more recent major version. Module consumers may even be totally unaware of later major versions.

Discoverability is a key issue. The mechanisms for module authors to advertise recent major versions are inconsistent, and can be low-visibility (documentation? README.md?) or highly disruptive (printing deprecation warnings in init, broken builds to force an investigation).

Abstract

We propose two improvements: one targeted at module consumers, and the other at producers.

For consumers, we propose a mechanism that notifies users of the latest major version of a module dependency when that dependency is first added to a project.

For producers, we propose adding a deprecated directive to go.mod files to signify the end-of-life of a major version.

These are just preliminary ideas which we hope to refine and improve in response to feedback gathered here.

Proposal 1: Notification of latest major version

We propose notifying users of new major versions when:

• They first add a requirement for an old major version of a module
• They update a requirement for an old major version of a module

There are a few ways users add requirements to their modules:

• By running go get github.com/user/repo
• By adding a require github.com/user/repo line to their go.mod file manually
• By adding an import line to a Go source file, and having goimports or gopls editor integration add the import line on save

For the latter two, the module isn't fetched until the go command is invoked within the module.

There are a few ways users update requirements:

• By running go get [-u] github.com/user/repo
• By running go get -u ./... from the module root
• Or to find out about available updates, they may run go list -m -u all

In each of these cases, the go tool plays a key role, and so we propose to make the go tool print a note if a requirement is being added when there is a more recent major version of the module available.

Examples

Consider a user fetching peterbourgon/ff with go get. We propose adding a notification to the output, alerting the user to a new major version:

$ go get github.com/peterbourgon/ff
go: finding github.com/pelletier/go-toml v1.6.0
go: finding gopkg.in/yaml.v2 v2.2.4
go: finding github.com/davecgh/go-spew v1.1.1
go: downloading github.com/peterbourgon/ff v1.7.0
go: extracting github.com/peterbourgon/ff v1.7.0
go: note: more recent major versions of github.com/peterbourgon/ff are available     👈
go: note: to install the most recent one, run `go get github.com/peterbourgon/ff/v3` 👈

Consider a user listing all of the most recent versions of their dependencies. We propose adding the latest major version alongside any new minor or patch versions:

$ go list -m -u all
example.com/my-module
github.com/BurntSushi/toml v0.3.1
github.com/mitchellh/go-wordwrap v1.0.0
github.com/peterbourgon/ff v1.6.0 [v1.7.0] <v3.0.1>                        👈 ONE OF
github.com/peterbourgon/ff v1.6.0 [v1.7.0] <github.com/peterbourgon/ff/v3> 👈 THESE
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7 [v0.0.0-20191204190536-9bdfabe68543]
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 [v1.0.0-20200227125254-8fa46927fb4f]
gopkg.in/yaml.v2 v2.2.2 [v2.2.8]

If a new requirement is added to a go.mod file manually, the go tool would print the notification when it first fetches the new module, as part of a go build, go test, etc. run.

Proposal 2: A new deprecated directive

Producer-side deprecation feature: add a deprecated line, which will cause the module to fail to fetch at that version, printing an error with the most recent non-deprecated version of the same major version.

module github.com/user/repo

deprecated

require (...)

If the go tool were asked to fetch this deprecated module, it would fail:

error: module github.com/user/repo@v1.7.1 is deprecated

If the Proposal 1 were adopted, the error message could be more useful:

error: module github.com/user/repo@v1.7.1 is deprecated
error: try the latest major version: go get github.com/user/repo/v2

The deprecated directive may optionally include a successor module identifier. If specified, the error printed when fetching this module version would also include a reference to the successor module. This can be used to point to the next major version:

deprecated github.com/user/repo/v2

Or to a new import path altogether:

deprecated gitlab.com/newuser/newrepo

If the go tool were asked to fetch this deprecated module, it would fail with a more explicit suggestion to the user:

error: module github.com/user/repo@v1.7.1 is deprecated
error: try its successor: go get github.com/user/repo/v2

If the successor module is also marked as deprecated and includes a successor module, the go tool might follow those links, and print the final non-deprecated module in the error message.

When package producers decide that a major version is deprecated, the intent is for them to create a new minor or patch release of that major version with the deprecated directive in the go.mod. This version will only be selected by the go tool when the corresponding major version is first added to a project, or if someone tries to update from an earlier version of that major version. In both cases, the go tool fails to fetch the module and provides a useful and actionable message: the user is instructed to pick a non-deprecated version.

Users can always use earlier versions of that major version, and MVS should ensure that the deprecated version will not be selected. If module A has a requirement for B at v1.7.0, and B v1.7.1 is later tagged as deprecated, A can continue to use B at v1.7.0, and the maintainer of A will only become aware of the deprecation when they try to update B.

It should not be possible for the go tool to mechanically add a deprecated version to a go.mod file.

A major version may be "un-deprecated" by publishing a subsequent minor or patch version without a deprecated directive in its go.mod. The specific deprecated version remains unusable, but the earlier and later versions still work normally.

Examples

A producer deprecating v1 of their module:

• Producer tags github.com/user/repo v1.0.0, v1.1.0, ... up to v1.7.0
• Producer tags github.com/user/repo/v2 v2.0.0, and soon afterward v2.0.1
• Producer adds a deprecated or deprecated github.com/user/repo/v2 line to the go.mod of the v1 branch and tags it v1.7.1

A consumer using a module that is then deprecated:

• Consumer has a module requirement require github.com/user/repo v1.7.0
• Producer tags the deprecated v1.7.1 version
• Consumer continues to use v1.7.0 without issue
• If the user tries to upgrade the dependency, the deprecation prevents it
• The user can keep their existing v1.7.0 or migrate to the new v2.x.y

A consumer tries to fetch a module at a deprecated major version:

• Consumer runs go get github.com/user/repo as a new dependency
• go get selects the highest version in the v0 or v1 trees, which is v1.7.1
• go get prints an error message:

error: module github.com/user/repo@v1.7.1 is deprecated
error: try the latest major version: go get github.com/user/repo/v2

• Consumer runs the suggested command, installing v2 of the module

Composition

One of Go’s strengths is the orthogonality of its features. We believe the proposed features compose nicely and strengthen each other.

Taken separately, the proposals can stand on their own: P1 provides package consumers with useful information without direct action from package producers; P2 allows package producers to give their consumers more specific guidance on an opt-in basis.

Taken together, the proposals enrich each other: P1 improves the error messages that accompany P2; P2 funnels users into the upgrade paths created by P1.

Integrations

pkg.go.dev

The Go package discovery website at pkg.go.dev shows modules and their versions. However, it obscures successive major versions when they exist, apparently treating major module versions as completely distinct. For example, the landing page for peterbourgon/ff shows v1.7.0 with a "Latest" bubble beside it. The versions tab does list other major versions, but under the heading "Other modules containing this package", which is confusing.

Instead, pkg.go.dev could feature a prominent indicator on the landing page for a v0/v1 module that there are two successive major versions (v2 and v3), to funnel the user toward the latter.

Editor integration (gopls, goimports)

Go text editor integrations typically include a feature that automatically adds import statements to source files based on the mentioned identifiers. Because of Semantic Import Versioning, this also gives those tools the responsibility of choosing the major version of the imported module. In the case where there is no suitable existing requirement in the project’s go.mod file, these editor integrations could alert the user to the availability of newer major module versions. How this works is outside the scope of this proposal.

Appendix

• Related proposals:
  • retract directive
  • go.dev: don't imply that vN.x.y is the latest version when vN+1 exists
  • go/list adds another flag to support showing major upgrade available for module

[bcmills]

Finally, note that we already have a Deprecated comment convention for identifiers within a package.

It would probably not be terribly difficult to write a tool that adds such a comment to every exported identifier of every package within a given module. That would allow existing tools that already understand deprecation (such as some IDEs?) to surface that information in a way that is friendlier to the user than breaking go get -u.

[neild]

It would probably not be terribly difficult to write a tool that adds such a comment to every exported identifier of every package within a given module. That would allow existing tools that already understand deprecation (such as some IDEs?) to surface that information in a way that is friendlier to the user than breaking go get -u.

It should be sufficient to add a Deprecated: comment to the package doc comment. I believe some (most? all?) tools which surface deprecation notices on symbols will also pick up a package-level deprecation notice.

Still, having the go tool surface module-level deprecation notices seems useful to me. Doing so at go get time seems like an opportune point for a user to receive the notice.

[jayconrod]

+1 to what @bcmills and @neild have said about not breaking go get -u and about // Deprecated: comments.

I'll add that the main way module retractions (#24031) will be surfaced to the user is through a warning in go get. If the build list contains a retracted version after go get has finished changing go.mod, go get will print a warning. It seems reasonable to extend that for deprecated modules.

[adg]

Thanks for the feedback everyone. We've refined and split this proposal into two new ones.

Please see the new proposals #40357 and #40323 that supersede this one.