proposal: cmd/go: list deprecations to include abandoned packages, as well #68316

Open TheCoreMan opened 4 months ago

Proposal Details

This is a proposal lifted from our discussions over at Cup o' Go, and is a direct follow-up of cmd/go: list deprecations and newer available major, minor versions by mitar.

Proposal: Highlight Deprecated & Abandoned Packages

This proposal aims to implement a system to highlight packages that have been marked as abandoned by their maintainers, and/or marked as such by a trusted third party. The goal is to improve package discovery and maintenance by clearly indicating modules that you can't really rely on anymore and suggesting potential alternatives.

Key Features

Marking "repo archived" on GitHub, GitLab, etc.:
- Automatically detect if a GitHub/GitLab repository is archived.
- Mark the package as abandoned in the Go documentation and package registry.
Third-party registry for deprecated packages (Allow for the third party - not necessarily maintain it within the Go team):
- Allow passing a config/flag for --package-fork-follow-server=https://somewhere/on/the/web
- Establish a third-party registry API, so maintainers can list deprecated packages and their suggested alternatives.
  - This will open up opportunities for SSDLC companies to set up their own registries to follow "the good forks" in their opinion
- The registry must provide an API for querying deprecated packages and their alternatives.

Benefits

Improves developer awareness of abandoned packages.
Encourages maintainers to clearly indicate abandonment status and suggest alternatives themselves, allowing for more responsible project sunsetting and better open-source well-being.
Facilitates smoother transitions to maintained forks or just plain better alternatives.

Example Workflow

A maintainer decides to abandon their module, because they're moving to Mars.
They archive the repository on GitHub.
The Go package index and documentation are updated to reflect the abandonment.
Developers using the package receive warnings and suggestions for alternatives during their workflow when running go mod outdated (or whatever subcommand we end up choosing in the original proposal), perhaps with a flag e.g. go mod outdated -include-abandoned.
The third-party registry provides additional context and options for finding maintained packages, e.g.:

> export GO_THIRD_PARTY_FORK_FOLLOWER=https://some.company.com/go-forks
> go mod outdated -include-abandoned -suggest-forks
github.com/thecoreman/some-package has been abandoned => replace with github.com/flimzy/some-package

Related Issues

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

As mentioned in https://github.com/google/deps.dev/issues/99 this could be something that's worth surfacing in deps.dev for wider use-cases, and then integrating the go toolchain directly with the API to retrieve the data.

I'm not sure if it'd make sense for us to do that (i.e. would this be the first time we add an external integration as part of the toolchain?) or if it's something we would want to implement separately here.

I +1 that this information is incredibly useful (having built tooling to do this myself) and that if we can have this surfaced in a central manner, this can make it easier for teams to respond to unofficial deprecations/abandonment of projects

If maintainers want to push developers away, they already have such a mechanism: the Deprecated comment in packages and modules.

This leaves the "trusted outsider" view, but that seems to push software development towards an endless treadmill of changes, rather than allowing for something to be "complete" and stable.

I feel like we already have that for when it's important: when a vulnerability is discovered, reported, and verified, it gets picked up by security tooling like golvulncheck. To that point, "abandonment" could just be a low severity entry in a third party vulndb, there's already an API and the ability to use a non-default server in the tooling.

If maintainers want to push developers away, they already have such a mechanism: the Deprecated comment in packages and modules.

Very true - but for instance even the Google Go team haven't done so with https://github.com/golang/mock and there are a number of other popular libraries doing the same.

To that point, "abandonment" could just be a low severity entry in a third party vulndb, there's already an API and the ability to use a non-default server in the tooling.

Interesting - how would a case i.e. the Gorilla Toolkit archiving, and subsequent unarchiving look?

I.e. a new GHSA-... would be created to note that any versions of gorilla/mux are affected by a low-severity CVE (repo is unmaintained) and then when the repos were picked up by a new maintainers team, we'd redact the CVE?

I understood the idea of using the vulnerability system for this as a response to the idea of "trusted third-parties" providing opt in checking for unmaintained packages, which I think would mean:

The trusted third party implements their own instance of the Go Vulnerability Database API, which describes each unmaintained module as having a "vuln".
Anyone that wants to rely on that third party's signals would run govulncheck with the -db option referring to the third-party implementation of the API.

This seems like it would technically work, though it does seem a little awkward to model "unmaintained" as a vulnerability because it's typically a property of a module as a whole rather than a specific version of a module, whereas vulnerabilities are treated as applying to a subset of available versions.

I guess in cases like the gorilla/mux example (where it became maintained again later) the module would initially have its vulnerability published but then the vulnerability would have its "fix version" retroactively set to the earliest available version of that module, so that then no version of the module would be considered "vulnerable". Or, I suppose, the vulnerability could be removed from the database entirely, if the tools used to consume it are designed to treat that as a retraction of the advisory. (I don't know if they are or not.)

(If I misunderstood what that idea was aiming at then I apologize.)

it's typically a property of a module as a whole rather than a specific version of a module, whereas vulnerabilities are treated as applying to a subset of available versions.

It may seem slightly awkward, but modules do exist where they are fundamentally broken, or have had a vulnerability since its conception. I still think it fits the model though, vulnerabilities represent potential risks, and lack of future updates is a risk that applies to all existing versions of the code.

Vulnerabilities are exposed in OSV format, there's a field for withdrawn https://ossf.github.io/osv-schema/#withdrawn-field

cc @ianthehat

This really seems out of the scope of the go command, but perhaps other tooling might want to check for modules in archived repos and advise users about them?

golang / go