proposal: cmd/go: add go mod verify -tag

[Esra-Al]

Proposal Details

Go lacks a CLI to verify local repositories against the Go checksum database. Adding a flag in go mod verify to check local git tags against the sumdb would help module authors ensure their contents haven't been tampered with. This could prevent issues from unauthorized changes by someone with force-push access to GitHub, GitHub itself, or even Google.

I propose adding a -tag flag accepting the following values

all: Check all local git tags against the sumdb. latest: Check only the latest local git tag [version]: Check a specific version (e.g., go mod verify -tag=v1.0.0).

-tag=latest is especially useful as the first command to run after pushing a new tag, as it will have the side-effect of loading it in the sumdb while checking it matches the local repository.

[gabyhelp]

Related Issues and Documentation

• proposal: cmd/go: add go mod verify -tag #68668 (closed)
• cmd/go: go mod verify should check hashes in go.sum against GOSUMDB. #47752

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[ianlancetaylor]

CC @matloob

[seankhliao]

isn't this just GOPROXY=direct go get my.module ?

[FiloSottile]

I'm pretty excited about this, so far we don't have a good story for how the author-to-sumdb link is secured, but it's a very small gap to fill thanks to the sumdb design. It also gives a canonically good answer to "how do I review a dependency knowing it's what will be used by my application" that doesn't involve vendoring or digging into the modcache.

isn't this just GOPROXY=direct go get my.module ?

Almost! That downloads the module from e.g. GitHub where it might have been tampered with. This checks the local copy as it was developed / is being reviewed. It's kind of like a hypothetical GOPROXY=local with nicer UI.

[gopherbot]

Change https://go.dev/cl/596097 mentions this issue: cmd: add go mod verify -tag

[Russ741]

Do we want go mod verify -tag to also verify the cache like go mod verify, or to skip that part?

[matloob]

cc @ianthehat Do you have a perspective on this?

[ianthehat]

I think that the functionality is a really good idea, and we should have it.

The ability to verify that the code you intended to make up a release flows through your code hosting site, into the module proxy and back down to you without modification is an important property to be able to check, and doing so is currently far too hard.

I am not totally convinced that go mod verify -tag=??? is the right interface to it, but am also happy to defer that decision to others. If it is then the help message needs more substantial rewriting than that cl (it starts with Verify checks that the dependencies of the current module, which are stored in a local downloaded source cache, have not been modified since being downloaded, and now we are making it do far more than that) We should also think if there are any other mod properties that we might want to verify in the future, and make sure the design allows for them if so.

[rsc]

This proposal has been added to the active column of the proposals project and will now be reviewed at the weekly proposal review meetings. — rsc for the proposal review group

[aclements]

Just for my own understanding, can someone (@ianthehat or @Esra-Al ?) explain how this would be used? It sounds like this would slot into a release process, but I'm not really sure where. Does this have to be something that's opt-in, or is there any way we can make it more automatic? What are the potential consequences of a user not doing this verification?

[ianthehat]

We probably want @FiloSottile to chime in here. My guess is that the most important use is at the same point that somebody tries to tag a repository they also want to verify that the whole world sees the code they had locally when that tag is fetched. For the majority of repositories that is probably an individual developer doing it, but it could equally be a release process, or a github action, anywhere that somebody has a local copy of the repository they believe is authoritative and wants to check the full round trip. Because we don't have tooling over actually setting tags, I don't see any part of current workflows where the go command knows it should check it. We did once talk about adding functionality to help users with tagging releases (like telling them if they should do a major or minor etc), but we never moved any further with it.

[Russ741]

Yeah, my understanding basically meshes with what @ianthehat posted - it can be incorporated into the release process and run to verify each new version shortly post-release.

Note that there's another use case for manual invocation here - if someone is doing a security audit of version x.y.z of a package, they can use this tool to verify that the version of the code they've checked out for inspection is the canonical x.y.z according to the go module mirror/checksum DB, rather than whatever is tagged as x.y.z in GitHub or wherever at the moment.

As far as further automation, the options I've come up with are well outside of this scope:

• it might be possible to have the go module mirror periodically recheck that its cached versions of a module match what the source repo is serving up and flag mismatches (not sure what it should do about them, though)
• it might be possible to set up some sort of "there's a new version of package foo.bar in the module mirror" feed for developers to subscribe to, so they find out about unexpected/malicious releases right away rather than when they initiate a valid release with the same version

[aclements]

Thanks. This seems well-motivated from a security perspective, but I think it's less clear what the exact user interface to this should be. This dovetails with the question of exactly when someone/something would run this verification command. Pinging @matloob

[matloob]

@ianthehat I'm interested to hear (read?) why you think go mod verify -tag may not be a good interface for this.

@Esra-Al I think if we do use go mod verify -tag as the interface, then providing the -tag flag without arguments should still do something. Could we have it do the all behavior? How much slower is all vs. latest on a repo with a lot of tags?

It seems like the alternatives to verify -tag would be (1) to implement the functionality in another tool in an x/ repo or (2) to implement it in another go mod subcommand. The main issue with (1) is that some of the vcs logic would have to be reimplemented/copied. The main issue with (2) to me is that if we do find other properties to verify, we would then end up adding new subcommands for each of them. I think there is some precedent for this type of interface with go clean which has a different behavior with and without the flags for cleaning various caches.

[aclements]

Ping @ianthehat for @matloob 's question above.

[ianthehat]

I am not convinced it is the right place because it has very little to do with mod files, and nothing to do with verifying a modules dependencies (which is what go mod verify does). It feels like it is just being elbowed into a fairly arbitrary spot just because that spot mentions module verification, rather than a place it actually belongs. On the other hand I don't have a good suggestion for where it really should go, or what other features might eventually join it. If we ever add features that help a user pick the right tag (we talked about this at one point, I think the suggesion at the time was go mod release), or even apply that tag, then it would kind of fit with those.