extended forwards compatibility for Go

[rsc]

Many people believe the go line in the go.mod file specifies which Go toolchain to use. This proposal would correct this widely held misunderstanding by making it reality. At the same time, the proposal would improve forward compatibility by making sure that old Go toolchains never try to build newer Go programs.

Define the “work module” as the one containing the directory where the go command is run. We sometimes call this the “main module”, but I am using “work module” in this document for clarity.

Updating the go line in the go.mod of the work module, or the go.work file in the current workspace, would change the minimum Go toolchain used to run go commands. A new toolchain line would provide finer-grained control over Go toolchain selection.

An environment variable GOTOOLCHAIN would control this new behavior. The default, GOTOOLCHAIN=auto, would use the information in go.mod. Setting GOTOOLCHAIN to something else would override the go.mod. GOTOOLCHAIN=local would force use of the locally installed toolchain, and other values would choose specific releases. For example, to test the package in the current directory with Go 1.17.2:

GOTOOLCHAIN=go1.17.2 go test

As part of this change, older Go toolchains would refuse to try to build newer Go code. The new system would arrange that normally this would not come up - the new toolchain would be used automatically. But if forced, such as when using GOTOOLCHAIN=local, the older Go toolchain would no longer assume it can build newer Go code. This in turn would make it safe to revisit and fix for loop scoping (discussion #56010).

See my talk on this topic at GopherCon for more background.

See the design document for details.

[rsc]

Updated link to design doc: https://go.dev/design/57001-gotoolchain.

[ianlancetaylor]

I just want to note that the potential for confusion here is high. go build and go install should report whenever they invoke a different toolchain.

[rsc]

I'm not sure about that. What I hope will be a common mode of usage is that you install some update-aware Go toolchain on your machine and then from that point on just edit go or toolchain lines in your go.mod in various projects and never explicitly update the locally installed Go toolchain ever again. The Go toolchain becomes a detail managed inside go.mod just like all the other inputs to your build. In that mode of usage, this reporting would print on literally every go command that gets run, which is too noisy.

[dominikh]

I think the combination of auto-upgrading and not allowing old toolchains to build new code will change the Go ecosystem to either no longer support multiple versions of Go, or to delay experimenting with new language features.

Where previously a Go module could target Go 1.21 but make use of 1.22 features in optional, build-tagged files, it can no longer do so. To be allowed to use Go 1.22 language features, the go.mod has to specify go 1.22, but this triggers auto-upgrading / build failures on Go 1.21. The module now only supports Go 1.22.

Auto-uprading makes this seem fine at first: anyone who needs Go 1.22 will get it for free, automatically. However, I don't think auto-upgrading can be assumed to be pervasive. For one, most people who hack on Go at least occasionally will be using GOTOOLCHAIN=local, pointed at a git checkout. Furthermore, I predict that some Linux distributions will patch their Go packages to disable the automatic upgrading, as it side-steps their packages and some are allergic to that. This'll lead to users who are left behind even more than they're now due to slow-moving distributions.

[zikaeroh]

I personally have a few reservations about this proposal.

Firstly, I don't feel that the Go version in go.mod should be the "minimum" supported Go version for a module. The way I have understood it has been that it's a "feature unlocker". See https://twitter.com/_myitcv/status/1391819772992659461, https://twitter.com/marcosnils/status/1391819263325974530, https://github.com/golang/go/issues/46201

It's very possible to enable a newer version of Go in go.mod to use things that are only available in that version, but gate those uses behind build tags, falling back to something else for older versions of Go. For example, //go:embed can be placed behind build tags such that it won't be used if the version of Go in use can't support it. The same applies for new exported stdlib types/functions. If go.mod enforced a minimum, that code couldn't be written (or at least be annoying to test).

This pattern seems common enough; x/tools and x/tools/gopls both set go 1.18, but in actuality test and support versions back to Go 1.16. It seems like this setup wouldn't be very tenable with this proposal implemented, as Go would automatically ignore that and download something else. Sure, maybe they could use GOTOOLCHAIN to work around that, but I think that'd be a major shift in how people generally get Go in CI. (It's also awkward to be able to go forward by changing Go on $PATH, but not backwards.)

Secondly, the automatic download/execution of binary releases of Go seems really surprising. I feel like it's going to be very awkward for Linux distributions to lose control of the toolchain in use without environment variables (especially if they patch Go). I do wonder how many distros might patch Go entirely to force GOTOOLCHAIN=local as the default. I believe there are also examples of corporate forks of Go (I've seen Microsoft's mentioned before), and those would also likely patch away this behavior becuase it'd be a bad thing for those to start bypassing the expected toolchain, especially without any sort of warning message that it's happening.

There are also systems where the binaries downloaded from golang.org won't be functional, e.g. Nix/NixOS (where the system layout is entirely different and outside binaries require a lot of work to use), musl-based distros like alpine (musl is not a first class libc for Go, as far as I know), and so on. Those distributors may also have to disable this download functionality to prevent breakage (be it to make things work consistently in the first place, or to just stop users from reporting the failures as distro bugs).

This part of the proposal is being compared to module dependency management itself. I strongly feel that modules work really, really well in comparison to other languages' package management scenarios (like node_modules, virtualenv, etc) in that it all happens automatically for you. But, I think the way this proposal does this for Go versions itself is going to be too surprising and likely to break.

(I mentioned some of this in https://github.com/golang/go/discussions/55092#discussioncomment-3755585, but my thread seems to have been missed as all of the threads around it were replied to.)

[rsc]

I spoke to @ianlancetaylor about his concerns. For most commands, you can always run go version to see what version you are going to get in the next command. I think that will be enough for those commands (Ian does not completely agree but is willing to wait and see). The one exception is go install path@version.

For go install path@version, I think we probably have to do the module fetch and then potentially re-exec the go command based on the go line. I would be inclined to say we don't respect any toolchain line, just as we don't respect any replace lines. That is, today go install path@version is shorthand for:

mkdir /tmp/empty
cd /tmp/empty
go mod init empty
go get path@version
go install path

I think it should continue to be shorthand for that, which would mean ignoring the toolchain line in path/go.mod.

In the case where go install path@version needs to fetch and run a newer Go toolchain, I think it would be appropriate to print a message:

$ go install path@version
go: building path@version using go1.25.1
$ 

[rsc]

@dominikh

I think the combination of auto-upgrading and not allowing old toolchains to build new code will change the Go ecosystem to either no longer support multiple versions of Go, or to delay experimenting with new language features.

Where previously a Go module could target Go 1.21 but make use of 1.22 features in optional, build-tagged files, it can no longer do so. To be allowed to use Go 1.22 language features, the go.mod has to specify go 1.22, but this triggers auto-upgrading / build failures on Go 1.21. The module now only supports Go 1.22.

This is true. Older versions of the module will still support Go 1.21, of course. It's just the latest version of the module that only supports Go 1.22. I don't think it's a sure thing that this is a problem. The same happens today for dependencies, of course, and it seems to be fine.

I agree that making it easier to update to a new Go toolchain may well result in people updating more quickly.

Auto-upgrading makes this seem fine at first: anyone who needs Go 1.22 will get it for free, automatically. However, I don't think auto-upgrading can be assumed to be pervasive. For one, most people who hack on Go at least occasionally will be using GOTOOLCHAIN=local, pointed at a git checkout. Furthermore, I predict that some Linux distributions will patch their Go packages to disable the automatic upgrading, as it side-steps their packages and some are allergic to that. This'll lead to users who are left behind even more than they're now due to slow-moving distributions.

People who hack on Go will be using GOTOOLCHAIN=local (that will be the default in their builds), but they will also presumably be hacking on the latest Go version, which will not be too old for any code.

I agree that some Linux distributions are likely to patch Go to default to GOTOOLCHAIN=local. That seems fine too, as long as users can still go env -w GOTOOLCHAIN=auto.

[rsc]

@zikaeroh, Alpine should not have problems running standard Go distributions starting in Go 1.20. I am not sure about NixOS. The only thing it should need is a libc.so.6 for the dynamic linker to resolve. Or maybe we should build the distribution cmd/go with -tags netgo and then it wouldn't even need that. I wonder what that would break...

[zikaeroh]

My (rudimentary) understanding is that libc.6.so is not located in the "typical" location on NixOS, so would not be found by a normal Go binary. It'd be at a long path like /nix/store/ikl21vjfq900ccbqg1xasp83kadw6q8y-glibc-2.32-46/lib/libc.so.6 as each package deterministically uses specific other packages. So, a flavor of the Go package that uses the precompiled binary releases of Go would use patchelf to fix this. Downloaded versions would not have that patching applied, and therefore would not work.

See also: https://nixos.wiki/wiki/Packaging/Binaries

[rsc]

@zikaeroh

This pattern seems common enough; x/tools and x/tools/gopls both set go 1.18, but in actuality test and support versions back to Go 1.16. It seems like this setup wouldn't be very tenable with this proposal implemented, as Go would automatically ignore that and download something else. Sure, maybe they could use GOTOOLCHAIN to work around that, but I think that'd be a major shift in how people generally get Go in CI. (It's also awkward to be able to go forward by changing Go on $PATH, but not backwards.)

x/tools/gopls only tests that far back because they want to build with what's on people's machines. If what's on people's machines knew how to fetch a newer toolchain then we'd have stopped needing to support older versions long ago.

I think that one reason people are slow to update to new Go versions because it is too difficult. What's difficult is managing Go installations. This proposal removes that difficulty, which in turn should make it easier for people to keep up with newer versions.

[rsc]

Regarding Linux distributions setting GOTOOLCHAIN=local by default (which I think would be fine), I'm curious whether they apply similar rules to rustup or nvm. Does anyone know?

[zikaeroh]

I think that one reason people are slow to update to new Go versions because it is too difficult. What's difficult is managing Go installations. This proposal removes that difficulty, which in turn should make it easier for people to keep up with newer versions.

I guess I'm confused; my impression was that the hardest problem in upgrading was not obtaining a new version of Go, but making sure that the Go code works for that new version of Go, which is the backwards compatibility proposal (not this one).

I would be very interested to know what proportion of Go users obtain Go via a package manager (versus golang.org). It seems to me like most package managers are going to set GOTOOLCHAIN=local (as noted by @dominikh and me above). But it also seems to me like most Linux users are getting Go via their distro's package manager and macOS users via brew (and I personally use scoop/chocolately/winget on Windows).

If all of those users are going to end up getting local as their default, is anyone going to use auto?

I'm curious whether they apply similar rules to rustup or nvm. Does anyone know?

On Arch, you can either install the rust package or rustup; both "provide" rust, but the actual packages are usually built in a clean chroot and automatically get the former, the latest rust compiler package.

Arch doesn't package nvm or any other node version manager; without the AUR, only the latest node/npm/yarn are provided. But, I think it's the case that many users may end up installing nvm (or its many alternatives), in which case the version in use is basically whatever anyone wants. The closest example I can think is projects which check in .node_version or use volta to pin a particular version for their project, but I've more often seen this for pinning a local dev setup and then more versions are checked in CI anyway.

[ianlancetaylor]

We should describe what happens if we drop an existing port (per https://go.dev/wiki/PortingPolicy). Presumably the go command 1.N will see "go 1.N+1", try to download the binaries for 1.N+1, fail, and then give an error and stop the build.

For cases like NixOS I think we would have to expect users on that system to set GOTOOLCHAIN=local. And that in turn suggests that perhaps there should be a way to build a toolchain such that GOTOOLCHAIN=local is the default if not overridden by the environment variable.

[rsc]

This proposal has been added to the active column of the proposals project and will now be reviewed at the weekly proposal review meetings. — rsc for the proposal review group

[rsc]

I filed https://github.com/golang/go/issues/57007 for building cmd/go without libc.so.6, which I think would make NixOS happy.

I agree that it should be possible to build a toolchain with GOTOOLCHAIN=local as the default, but I don't think most package managers should do this. For example I don't think it makes sense for user-installed package managers like Chocolatey or Homebrew to do this at all. They are not as pedantic about "we are the only way to install software on your machine!" as the operating system-installed package managers are, and they should not be going out of their way to break what will end up being a core feature of the Go experience.

I also think a fair number of Go developers still use the installers we provide, and those of course will have the GOTOOLCHAIN=auto default.

[rsc]

We should describe what happens if we drop an existing port (per https://go.dev/wiki/PortingPolicy). Presumably the go command 1.N will see "go 1.N+1", try to download the binaries for 1.N+1, fail, and then give an error and stop the build.

Yes, exactly. And we can give a good error.

[rsc]

I guess I'm confused; my impression was that the hardest problem in upgrading was not obtaining a new version of Go, but making sure that the Go code works for that new version of Go, which is the backwards compatibility proposal (not this one).

Go's backward compatibility is already very good. To the extent that it needs work, the backwards compatibility proposal will make it even better. That will leave actually getting the upgraded Go toolchain as the hardest problem. My experience maintaining other machines where I build Go programs but don't do Go development has been that I don't upgrade often at all because it is annoying to go download and unpack the right tar files. If all it took was editing a go.mod, I would do that far more often.

Another point that I forgot to make in the doc is that setting the Go toolchain in go.mod means that all developers working on a project will agree on the toolchain version, without other special arrangements. This was of course one of the big improvements of modules and moving out of GOPATH, for dependency versions. The same property can be provided for the Go toolchain version. This would also mean that if you are moving back and forth between two projects that have chosen different Go toolchain versions as their standard toolchain, you get the right one for that project automatically by virtue of just being in that project. You don't have to change your PATH each time you switch, or maintain a global symlink in $HOME/bin, or remember to type 'go1.18 build' in one place and 'go1.19 build' in the other, or any other kludge. It just does the right thing.

[rittneje]

In a CI/CD environment, I don't think this feature would be useful. I would expect that job to be configured to use the correct Go version in the first place. Downloading a newer toolchain might not work anyway due to firewall restrictions. And as others have mentioned, using an older compiler with GOTOOLCHAIN=local should not fail, as it may mean the module only uses newer language features conditionally and is verifying compatibility. In addition, this could easily lead to a situation where a developer who attempts to test with an older go version on their local machine but forgets to set GOTOOLCHAIN=local (or doesn't know about it) will get different results than the build server.

On another note, while the go directive in go.mod controls language features like octal literals and generics, today it has no bearing on the standard library implementation. My expectation is that if I run go build with compiler version 1.X, then it will use standard library version 1.X. But with this change, that will not be the case if I use an older compiler, unless I set GOTOOLCHAIN=local.

Another point that I forgot to make in the doc is that setting the Go toolchain in go.mod means that all developers working on a project will agree on the toolchain version, without other special arrangements.

We build our own version of the toolchain and distribute it to our developers, rather than using what is on golang.org. So this would not address that problem for us, especially because it sounds like such a toolchain will default to GOTOOLCHAIN=local anyway. What would be more useful for us in this regard is a way to do an exact string match on the compiler's go version and fail if it is wrong. But this is independent of the go directive. (For example, we are currently using 1.18, but our go.mod file says 1.17 to prevent developers from using generics.) I see this is touched on with the new toolchain directive but it's not clear it would buy much in our particular use case.

Another use case is if I am writing a library and want to easily test it with multiple go versions on my local machine. It would be convenient if I could just do something like go test -go=1.17 ./... and have it test with the latest 1.17 release (regardless of what my "real" go version is). But the key is I'd want to specify on the command line, not in go.mod.

[zikaeroh]

Another point that I forgot to make in the doc is that setting the Go toolchain in go.mod means that all developers working on a project will agree on the toolchain version, without other special arrangements.

I find this to conflict with the idea from the original proposal that it's a minimum version; if my project says "go 1.13" because that's the minimum, I very likely do not want to use that version of Go for development. A newer toolchain will be faster and behave better when called by tooling like gopls or other analyzers (regardless of the version of Go that gopls is compiled with). Or, I will definitely want to publish binaries using the absolute latest version of Go possible. For example, esbuild says "go 1.13", but the actual binaries published to npm are from Go 1.19. If 1.13 this were to be the expected development version, that wouldn't be very optimal.

[mateusz834]

An environment variable GOTOOLCHAIN would control this new behavior. The default, GOTOOLCHAIN=auto, would use the information in go.mod. Setting GOTOOLCHAIN to something else would override the go.mod. GOTOOLCHAIN=local would force use of the locally installed toolchain, and other values would choose specific releases. For example, to test the package in the current directory with Go 1.17.2:

GOTOOLCHAIN=go1.17.2 go test

To be honest I am fine with: go install github.org/dl/go1.17.2@latest.

[rittneje]

If you have a module that says go 1.12 [...] Cloud Native Buildpacks will always build your code with Go 1.12, even if much newer releases of Go exist. [...] The GitHub Action setup-go [...] has the same problems that Cloud Native Buildpacks do.

I feel like this proposal also doesn't really address this problem. To me, it sounds like both of these features are fundamentally mis-designed. If I am publishing a library on GitHub, then I should be specifying a range of minor versions (e.g, 1.17+), and then it tests with the latest patch release of each of them as part of my merge gate. Since the go.mod file cannot be relied upon to denote the lower or the upper bound, it really has no bearing on this, except possibly as the default lower bound. Separately, if I am publishing an actual binary as a release artifact, then I should be specifying the Go version (for build reproducibility), but the go.mod should not be considered at all.

One final feature of treating the go version this way is that it would provide a way to fix for loop scoping, as discussed in discussion #56010. If we make that change, older Go toolchains must not assume that they can compile newer Go code successfully just because there are no compiler errors.

Existing versions of the Go compiler already do not fail just because the go directive is newer.

[ianlancetaylor]

if my project says "go 1.13" because that's the minimum, I very likely do not want to use that version of Go for development. A newer toolchain will be faster and behave better when called by tooling like gopls or other analyzers (regardless of the version of Go that gopls is compiled with).

I don't think this is a concern here. This proposal does not say that newer Go versions should download an older Go toolchain. It says that older Go versions should download a newer Go toolchain. When a newer Go version sees an older "go" line in go.mod, it will emulate the language features of the older Go language (that is already true today).

[zikaeroh]

don't think this is a concern here. This proposal does not say that newer Go versions should download an older Go toolchain.

I agree; that was my original interpretation of the proposal. It just seemed like the followups implied otherwise. (That is how pinning tends to work in other languages like node.)

[gopherbot]

Change https://go.dev/cl/450916 mentions this issue: cmd/go: draft of forward compatibility work

[beoran]

As others here have explained, this proposal would cause a lot of problems for CI/CD, for package management, for conditionally using newer features in older codebases with conditional compilation, ...

The version of go in go.mod is now a minimum, which is fine. Rather than a magic environment variable, I would add a new command, go upgrade, that can upgrade the go compiler if installed locally from the official packages, but gives a helpful message if one should upgrade using the package manager in stead.

[zikaeroh]

x/tools/gopls only tests that far back because they want to build with what's on people's machines. If what's on people's machines knew how to fetch a newer toolchain then we'd have stopped needing to support older versions long ago.

FWIW I only mentioned x/tools as it's an example of one the Go team maintains that was at the top of my mind. I don't find this particular point to be convincing as while this may be true for gopls specifically as it's an executable tool that users need to run somehow, most other examples are libraries (which are of course going to be built with "what's on people's machines"). If everyone is bumping the go directive in go.mod to gain access to new features, they still may be supporting older versions of Go (especially if they are following Go's own supported version policy).

This also is not exclusive to language features; I recall a series of CLs updating the x repos for lazy loading (#36460), for example CL 316111, which was only possible by bumping the version directive. All of the CLs in that series said:

Note that this does not prevent users with earlier go versions from successfully building packages from this module.

Which to me very much supports the idea that go directive is not a minimum version at all, but a feature unlocker.

(I apparently forgot to actually submit this reply days ago, oops.)

[mark-pictor-csec]

This seems like a bad idea to me.

Other text file formats include a line identifying the tool version that created the file, and do not imply that older versions cannot process the file properly. So the current behavior is not without precedent. (Examples of these other formats aren't springing to mind, but it wasn't a novel concept when I first saw it in go.mod).

This change assumes that upgrading the toolchain will always be beneficial and regressions will never happen. That stance doesn't seem much different to me than that of a certain OS, where users can be forced to reboot for an update - with no possibility to defer or skip the update. With a language rather than an OS, this behavior may be less infuriating and less disruptive for most users, but I think it's still unacceptable.

If a change is to be made, I would instead deprecate the go keyword and replace it with two, such as recommended_go and minimum_go. The former would have the same behavior as the go keyword currently has, while the latter would cause the toolchain to exit with error. Neither of these keywords would cause silent upgrades, and the names should be less likely to mislead.

If I understand correctly, the minimum_go keyword's behavior should allow for the for loop scoping fix mentioned in the description, as it'd put a lower bound on the toolchain version in use.

[rsc]

@mark-pictor-csec I think you just described the actual proposal.

minimum_go is spelled 'go'. It sets the minimum Go toolchain version that must be used to compile the module, whether used directly (via git clone + cd into it + run go commands) or indirectly (as a module dependency).

recommended_go is spelled 'toolchain'. It only applies when code is used directly by cd'ing into the module. It is ignored when the module is used indirectly as a required dependency.

It is not true that the proposal "assumes that upgrading the toolchain will always be beneficial and regressions will never happen." In fact I have gone out of my way in the past to explain why that's false. See for example https://research.swtch.com/vgo-mvs. Instead, the proposal, like Go modules as a whole, puts the user in control of which toolchain they use. There are no gremlins going around updating the Go toolchain behind your back. You only get a new go toolchain when you explicitly make a change to your go.mod file to indicate that. This is the same behavior as other modules: you keep getting the one listed in go.mod, even if there are newer ones, until you explicitly upgrade.

[rsc]

Re: potential confusion

A few people, including @ianlancetaylor, mentioned potential confusion. That potential definitely exists (indeed, many people on this issue are confused about the details, which suggests I did not present this well enough). We have to make it easy for people to understand what toolchain they are using and why. The way to do that has not changed: run go version and it will tell you.

There is only one time when go version does not help you answer the question, and that is when you are using go install path@version, which does not use the local go.mod. In this case, if the go command does not use its bundled toolchain, we should print a message, like:

$ go install path@version
go: installing path@version using go1.25.1
$ 

That message combined with the ability to run go version should take care of understanding what is happening. It does not directly address understanding why. For that, we need to document the rules clearly. Perhaps go help version would be a good place to document that. I would write something like:

The Go distribution consists of a go command and a bundled Go toolchain, meaning the standard library as well as the compiler, assembler, and other tools. The go command can use that bundled Go toolchain as well as other versions that it downloads as needed. Running “go version” prints the version of the bundled Go toolchain that is being used, which depends on both the GOTOOLCHAIN environment variable and the go.mod file for the current work module.

These are the rules the go command follows to decide which Go toolchain to use. The rules are listed in priority order, from highest to lowest: the first matching rule wins.

• If GOTOOLCHAIN=local, then the go command uses its bundled toolchain.
• If GOTOOLCHAIN is set to a Go version, such as GOTOOLCHAIN=go1.23.4, then the go command always uses that specific version of Go.
• If GOTOOLCHAIN=auto, then the go command consults the work module's go.mod file.
• If GOTOOLCHAIN is unset, it defaults to auto in release builds and to local when building from Go's development branches.
• If the go.mod file contains a “toolchain” line, the go command uses the indicated Go toolchain. The line “toolchain local” means to use the go command's bundled toolchain. Otherwise the “toolchain” line must name a Go version, like “toolchain go1.23.4”, and the go command uses that specific version.
• If the go.mod file contains a “go” line, the toolchain decision depends on whether that line names a version of Go older or newer than the bundled toolchain. If the named Go version is newer than the bundled toolchain, the go command uses that specific newer version. Otherwise, the go command uses its bundled toolchain, emulating the old toolchain version as needed.
• Finally, if GOTOOLCHAIN is unset and the go.mod has no “toolchain” or “go” line, the go command uses its bundled toolchain.

[rsc]

Re: losing control over which Go toolchain is used

@zikaeroh raised a concern about whether Linux distributions would “lose control of the toolchain in use”, and @dominikh wrote that he thought “some Linux distributions will patch their Go packages to disable the automatic upgrading.” @ianlancetaylor said something similar to me directly. @zikaeroh seemed to imply that even Homebrew/Chocolatey/etc might apply such a patch. I of course cannot predict what these systems will do, but I would encourage them not to start deleting or disabling features in the software they package. I expect toolchain version selection to become a standard part of the Go developer workflow, and it would be a shame for these packagers to make it hard for Go users to get their work done. In the end, it's easy enough to override with go env -w GOTOOLCHAIN=auto (unless the code is removed entirely!), but it would still be best for Go to be the same out of the box for all users, no matter which box it comes out of.

I used this framing in my previous comment:

The Go distribution consists of a go command and a bundled Go toolchain, meaning the standard library as well as the compiler, assembler, and other tools. The go command can use that bundled Go toolchain as well as other versions that it downloads as needed.

I think that framing is useful when thinking about Linux distributions and other packagers too. Just as the go command can fetch specific code dependencies for use in a build, it would now be able to fetch specific toolchain dependencies for use in a build.

To address what might be the concerns, being able to fetch specific toolchain dependencies for a specific build, does not overwrite the existing bundled toolchain provided by the operating system. Nor does it change what version of Go you get when you create an empty directory, run go mod init, and start hacking: you get the bundled toolchain provided by the packager. But if a user explicitly indicates that they want to build source code that is too new for the bundled toolchain (for example, the go.mod explicitly says go 1.25 and the bundled toolchain is only go 1.21), then instead of a build failure, the go command fetches and uses an appropriate newer toolchain. Similarly, if a user explicitly indicates that they want a specific toolchain by setting GOTOOLCHAIN=go1.19.2 or writing toolchain go1.19.2 in go.mod, the go command will fetch and use that toolchain as requested.

Linux distributions packaging commands written in Go already have to decide how to handle those commands' module dependencies:

• One option is to trust in Go's high-fidelity, reproducible builds and let the go command fetch the dependencies directly. I would hope that systems that take this approach are also comfortable letting the go command fetch any toolchain dependency as well, since the toolchain fetches have the same high-fidelity, reproducible behavior as module dependency fetches.
• The other option is to insist on managing and providing all the source code that goes into the build. This is a completely understandable choice for a packager, and I assume they have built up mechanisms for adjusting the build environment to do that. For example perhaps they set GOPROXY=off to ensure there are no downloads and they supply an appropriate go.work or go.mod filled with replace statements to provide all the dependencies. Even packagers with those custom builds don't default GOPROXY to off for ordinary use by users, at least as far as I know. They simply run their own builds in a non-standard environment. I would hope that systems that take this approach would continue to customize their own build environment but not leak those details into the packaging of Go for end users. It is also worth pointing out that setting GOPROXY=off to disable module dependency fetches will also disable toolchain dependency fetches, since toolchains are treated as modules.

In either case, I don't think the decisions that distributions packaging Go commands make for their own builds should leak into their own packaging of Go itself. Setting a default GOTOOLCHAIN=local in a packaged Go distribution would make builds break when the go line is too new, and it would also make the go command ignore any toolchain line in the current work module's go.mod. Both of these would be confusing to Go users, who will expect the behavior explained in Go's own documentation as well as any new books about Go that are written.

As I've noted before, it wouldn't really make sense to me for a distribution to default GOTOOLCHAIN=local unless they are also defaulting GOPROXY=off and also disallowing packages like rustup and nvm, both of which do the same thing that this proposal would have the go command do: manage a collection of toolchains and run the version specified by the user's configuration for a specific build.

I am only speculating at the concerns of the Linux and other packagers. I would be happy to hear from them directly about any concerns and work with them to address those in a way that works for them and for Go. On the Linux side, perhaps @stapelberg has some thoughts? And if anyone working on Homebrew, Chocolatey, or others would be inclined to default GOTOOLCHAIN=local in those packagers, could you please get in touch? Commenting here is fine, or rsc@golang.org. Thanks.

[zikaeroh]

minimum_go is spelled 'go'. It sets the minimum Go toolchain version that must be used to compile the module, whether used directly (via git clone + cd into it + run go commands) or indirectly (as a module dependency).

This is the part that I still fundamentally disagree with; there are many examples listed where this directive is being used to allow newer features in a module, without restricting it to compile with that version or higher, be it language features or go.mod/go.sum changes themselves.

If this proposal doesn't actually enforce this minimum and doesn't make it difficult to use typical CI tooling like setup-go, then that's fine; a new toolchain directive is certainly interesting. But my understanding based on phrases like:

for example, the go.mod explicitly says go 1.25 and the bundled toolchain is only go 1.21), then instead of a build failure, the go command fetches and uses an appropriate newer toolchain

Is that this minimum is being enforced, and I believe that to be undesirable in too many cases to make it feel like a benefit overall.

---

@zikaeroh seemed to imply that even Homebrew/Chocolatey/etc might apply such a patch. I of course cannot predict what these systems will do, but I would encourage them not to start deleting or disabling features in the software they package.

I'll retract this specific bit; these all use the binary releases of Go. E.g. homebrew, chocolatey, scoop, winget

It's the source-using packagers that I am mainly concerned about, i.e. basically all Linux distros, and Nix (which can be run on any distro, even macOS).

As I've noted before, it wouldn't really make sense to me for a distribution to default GOTOOLCHAIN=local unless they are also defaulting GOPROXY=off

Personally, I don't find this to be a good comparison; setting GOPROXY has effectively no impact on the build besides its performance. I never have to worry about the value of the proxy because no matter how I set it, I'm going to get the same result. The only concern would be where my traffic goes, which is a different discussion.[^1]

The toolchain is another story; this value can significantly affect my compile by downloading a large toolchain and using it, without my interaction or knowledge. It's a behavior that no other language has, without user involvement to either change the toolchain or install a software which does this explicitly.

[^1]: The distros I deal with (Arch) don't disable the proxy, though I believe that there are distros like Debian (maybe Fedora too?) who insist on packaging the source of all Go code into GOPATH and (seemingly) bypass the module system entirely.

[rsc]

@zikaeroh, I hear you about the change around "minimum" semantics. I am writing replies for one topic at a time and haven't gotten to that one yet.

Personally, I don't find this to be a good comparison; setting GOPROXY has effectively no impact on the build besides its performance.

This is true of GOPROXY=direct, but I said GOPROXY=off. The former means just use direct connections to the original source hosts. The latter disables downloading code from anywhere on the internet.

I believe that there are distros like Debian (maybe Fedora too?) who insist on packaging the source of all Go code into GOPATH and (seemingly) bypass the module system entirely.

I can believe they do this for packaging commands written in Go as Debian packages. I'm happy for them to do whatever is appropriate for their use cases there. I'm concerned here instead with what defaults users get for their own Go development. My understanding is that the proxy remains enabled in that case.

[zikaeroh]

This is true of GOPROXY=direct, but I said GOPROXY=off. The former means just use direct connections to the original source hosts. The latter disables downloading code from anywhere on the internet.

You're right, my mistake; my brain clearly wasn't functioning at this hour.

I can believe they do this for packaging commands written in Go as Debian packages. I'm happy for them to do whatever is appropriate for their use cases there. I'm concerned here instead with what defaults users get for their own Go development. My understanding is that the proxy remains enabled in that case.

Certainly, yes. I meant that to be an aside as to not make it the point of the comment.

[rsc]

@zikaeroh I am also confused about

The toolchain is another story; this value can significantly affect my compile by downloading a large toolchain and using it, without my interaction or knowledge.

What does "without my interaction or knowledge" mean? You only get a new toolchain if you explicitly ask for it by editing a go.mod or setting GOTOOLCHAIN. The only way this could happen without your knowledge is if you cd'ed into someone else's module after a git checkout and ran a go command. Understanding that when you work on someone else's project you may use a different toolchain is a mental model shift that we will have to make clear to users, but collaborating on a project is already a much closer relationship than most code use. I mentioned earlier how go install path@version may use a different toolchain, but it will inform you of that. If I'm misunderstanding you, can you give a specific sequence of commands that would trigger a toolchain fetch without your interaction or knowledge? Thanks.

It's a behavior that no other language has, without user involvement to either change the toolchain or install a software which does this explicitly.

This is a bit of a philosophical disagreement, I think. Go has never been held back by limiting itself to behaviors that no other language has. We wouldn't have a go command at all if we did that. We've always focused on the full Go environment not just a language compiler/interpreter.

For example, as a very loose analogy, Go has never aimed to ship /usr/bin/python but for a different language. It has always been more like python+pip+pytest+... instead. It's a full environment. We could just as easily say that no other language can download dependency modules without user involvement or installing software specifically for that task (pip, cargo, etc). We put that functionality in the main Go toolchain precisely because it is an critical part of a full environment. Managing the Go toolchain version as well is a little like adding +virtualenv to that list, although much of what virtualenv does is already taken care of by modules and static linking, so maybe it should be there already. Like I said, the analogy is only very loose. Perhaps a better analogy is that Go is more like rustc+cargo than just rustc, and with this proposal it becomes more like rustc+cargo+rustup. In any event, managing the toolchain version is another critical part of a full environment, and it doesn't bother me at all - in fact I think it is a good thing - for Go to include this functionality out of the box rather than expect people to discover and install a separate tool.

[rsc]

Re: whether its easy enough to manage multiple toolchain versions today

A few people have mentioned that they don't believe it needs to be more convenient to manage multiple Go toolchains. For example @mateusz834 wrote “To be honest I am fine with: go install github.org/dl/go1.17.2@latest”, and @beoran suggested perhaps we need a “go upgrade”. I don't believe those are easy enough.

There's a popular DevOps meme (in the old sense) about whether servers are treated like pets or cattle. There's a full writeup and history here but this is the key point:

In the old way of doing things, we treat our servers like pets, for example Bob the mail server. If Bob goes down, it’s all hands on deck. The CEO can’t get his email and it’s the end of the world. In the new way, servers are numbered, like cattle in a herd. For example, www001 to www100. When one server goes down, it’s taken out back, shot, and replaced on the line.

In the pets mode, people invest manual effort in maintaining each instance of the thing. In the cattle mode, at least the right way, people spend time on automating the maintenance and then it runs itself. This scales far better and it fundamentally changes what is possible in the system.

In the early days of GOPATH, every dependency was a pet. There wasn't even go get. You lovingly went out in search of a new dependency, brought it home, gave it a name, and got it settled in just the right place for its new life with in your file system. Introducing goinstall (which became go get) was a step away from that model: it took care of downloading the dependency, giving it its name (import path), and storing it in the right directory in your file system. Dependencies at that point were on their way to being cattle, but dependency versions were still pets. You had to go around to each dependency and adjust each to be exactly the version you wanted. And all builds on your system used the versions you had carefully selected, which were probably a different set from what other people had. Tools like godep and the line of tools that followed, eventually culminating in Go modules, moved versions from being pets to cattle. Now it's all automated, and no one has to go around making sure that the dependency versions on one particular machine match all the other machines being used for that project. And when you switch between two different projects, the right dependency versions get pulled into your builds automatically in each case. For example, as I write this, Kubernetes master is using github.com/emicklei/go-restful/v3@v3.9.0 while Istio master is using github.com/emicklei/go-restful/v3@v3.8.0. If I work in one project in the morning and the other in the afternoon, my go command automatically supplies the version appropriate to the project I'm working on at the moment. That even works if I'm running tests on both in different directories simultaneously.

We no longer have to think about which version of go-restful we have installed on each machine we use, but we still do have to think about which version of Go is installed. The Go toolchain is still a pet, one that requires ongoing maintenance on every machine where it is installed. It is great that we have golang.org/dl/go1.19.2, but that only gets me a command named go1.19.2. If I am working in two different projects that have two different standard Go toolchains they want developers to use, then I have to manage that myself, such as by changing my PATH when I switch from one to the other. And the idea of a “go upgrade” that upgraded “the” Go toolchain still assumes a single pet toolchain. This proposal is about automatic management of a herd of them instead.

As @zikaeroh pointed out, compatibility is another important problem for helping people upgrade to newer Go versions, and we are working on that separately (in #56986). But I can say that from my own experience that just the bother of needing to do the update is enough that many of my machines have very old Go versions installed. If I could just get the right one based on what my go.mod says, I'd be much happier. That's one less pet for me to take care of. This is also exactly why Cloud Native Buildpacks the setup-go action use the go line for toolchain selection.

I do appreciate that for some people, managing “the” Go toolchain on their machine is no big deal. For some people managing the specific checked out versions in GOPATH was also no big deal. These things tend not to be when they are small. But at scale it gets harder, and we are starting to see the scaling issues. Not everyone, but at least some people. In the discussion, for example, Josh mentioned that he uses a script that picks the Go version based on his go.mod. We should address the scaling issue for everyone, so that people don't have to build their own bespoke solutions. If people want to keep managing “the” Go toolchain on their machine (including forcing GOTOOLCHAIN=local if they are so inclined), that's entirely reasonable. But they shouldn't have to.

[zikaeroh]

If I'm misunderstanding you, can you give a specific sequence of commands that would trigger a toolchain fetch without your interaction or knowledge? Thanks.

I often clone other peoples' projects or switch between projects of my own. If I open any of those codebases, I may have to download an entire Go distribution before being able to do anything. That to me is very surprising. Unlike a typical Go module, the Go distribution is some 150 MB. If everyone uses the toolchain directive to specify which version is intended for development, that feels like a lot of versions of Go to download (not everyone has incredible internet) and cache locally, when it seems like I have always been fine using the absolute latest version of Go (something that I believe #56986 is intending to improve even further in that it addresses the question "how can we keep code working as intended when compiling with new versions of Go?").

This is a bit of a philosophical disagreement, I think. Go has never been held back by limiting itself to behaviors that no other language has. We wouldn't have a go command at all if we did that. We've always focused on the full Go environment not just a language compiler/interpreter.

I figured you'd say that, and I agree that this isn't a major criteria on the whole. My philosophy is just that if something "surprises" a user, that surprise should be positive. Modules are "surprising", but in the sense of "wow, I didn't have to npm ci when I changed branches, it just works!", an incredibly refreshing surprise and a major selling point I use when I talk about Go. The same applies to the module proxy, to testing, benchmarking, fuzzing, profiling, and so on.

But I think that the surprise provided by this proposal is not positive. I feel like I'm going to be surprised if I have to wait for a 150 MB download to start coding, end up with a huge module cache because many versions of Go itself are sitting there, and the potential to have accidentally compiled my project with an old version of Go because toolchain says that's what should be used (but my CI or Dockerfile was configured to use the latest). These to me feel like places that will annoy people at best, and really trip up someone in deployment or publish at worst.

[mateusz834]

@rsc What will happen when i try to go install sth-remote@latest and sth-remote go.mod has a toolchain directive? Is it going to download a toolchain and use it?

[rsc]

@mateusz834

What will happen when i try to go install sth-remote@latest and sth-remote go.mod has a toolchain directive? Is it going to download a toolchain and use it?

No. It will respect the go line but not the toolchain line (see https://github.com/golang/go/issues/57001#issuecomment-1332652062 for rationale). So if you have a new enough go already, it will use that. Otherwise it will use a newer toolchain and print a message to tell you. Actually it will print before it uses it. If using meant also downloading, you'd have an indication what was slow and could ^C.

I also think it is probably OK to print "go: downloading go1.19.2" the same way we print about go downloading modules today, so you'd get a download print in all cases where the toolchain is not already cached.

[rsc]

@zikaeroh thanks for the clarifications in https://github.com/golang/go/issues/57001#issuecomment-1339629395. I expect that most projects will use go without toolchain, in which case you should not end up with many different Go versions, especially if your pet toolchain is the newest possible one. We are also working on cutting the release down. Go 1.20 will be more like 95MB instead of 150MB to download, and we have ways to keep chipping away. We might be able to get as low as 50MB. That's still large but it's not enormous, especially compared to modern disks.

For projects using Docker and CI there has always been the opportunity for disagreement between your machine and CI. I hope those would migrate to respecting the toolchain line in go.mod (the usual "go mod download" layer caching trick would still work), precisely so that the disagreement goes away. On those kinds of projects where everyone cares about the specific toolchain being used, if Docker/CI use the toolchain in the go.mod and so do you, then disagreement between your machine and Docker/CI is a thing of the past. That's exactly one of the motivations for this change.

[mark-pictor-csec]

@mark-pictor-csec I think you just described the actual proposal.

Yes and no. The functionality provided by keywords may be the same (and apologies, I seem to have glossed over the toolchain functionality) but the names are different. My take is that go and toolchain impart less meaning than alternatives using words like minimum or recommended. Deprecating go and adding a new keyword with the same behavior but a more descriptive name serves to clarify what the keyword does (and doesn't) do.

If I missed out on this discussion and was confronted with a go.mod containing toolchain, I suspect that I'd assume it had the same meaning as go. I think the chances are quite low that I'd land on the correct meaning of the keyword or that I'd understand that go had a new meaning. Those sorts of false assumptions seem much less likely if words like minimum or recommended are in play :)

minimum_go is spelled 'go'. It sets the minimum Go toolchain version that must be used to compile the module, whether used directly (via git clone + cd into it + run go commands) or indirectly (as a module dependency).

recommended_go is spelled 'toolchain'. It only applies when code is used directly by cd'ing into the module. It is ignored when the module is used indirectly as a required dependency.

It is not true that the proposal "assumes that upgrading the toolchain will always be beneficial and regressions will never happen." In fact I have gone out of my way in the past to explain why that's false. See for example https://research.swtch.com/vgo-mvs.

Instead, the proposal, like Go modules as a whole, puts the user in control of which toolchain they use. There are no gremlins going around updating the Go toolchain behind your back. You only get a new go toolchain when you explicitly make a change to your go.mod file to indicate that.

I would argue that this user control is not always true. Consider: you are using an older (but still supported) go version. You have a module foo, importing someone else's bar; since the last time you updated the dependency, bar has been updated to require the latest version of go. If you update foo to use the latest version of bar, the go keyword will enforce a minimum version, so your current toolchain won't even try to build. Rather than exiting with an error, it sounds to me like the toolchain would download and run a new version. I strongly prefer being confronted with an error, rather than the tool (go) deciding to download and run a new version of itself for me.

This presupposes that the user has not throughly vetted the changes in bar and does not realize a newer version is required. That's a failing on the user's part, but it seems to me that downloading and running a new toolchain exacerbates their mistake.

[mark-pictor-csec]

Another issue I have with automatic toolchain downloads is that this is another link in the security chain. Hopefully never a particularly weak link, but no matter how strong it does give attackers more area to investigate.

A few years ago, I would've said the weaknesses I can imagine would require nation-state resources to exploit, but some ransomware groups have become quite rich. Stealing or forging certs may be within their reach, in which case an MitM attack becomes plausible: intercept module downloads, updating the required go version to require a new toolchain, and then intercept traffic to whatever server is providing the toolchain and provide a tampered version. Not a novel attack, but considerably easier if the download is automated. With a person in the loop, they have the opportunity to notice problems, for example by verifying checksums in a way that the attacker is unable to MitM or by wondering why a new toolchain release has broken the normal update cadence.

[rsc]

@mark-pictor-csec

I would argue that this user control is not always true. Consider: you are using an older (but still supported) go version. You have a module foo, importing someone else's bar; since the last time you updated the dependency, bar has been updated to require the latest version of go. If you update foo to use the latest version of bar, the go keyword will enforce a minimum version, so your current toolchain won't even try to build. Rather than exiting with an error, it sounds to me like the toolchain would download and run a new version. I strongly prefer being confronted with an error, rather than the tool (go) deciding to download and run a new version of itself for me.

What happens is something in the middle. A few releases back we made go get only adjust requirements, not also run builds. So when you run go get foo, it will update the version of foo, bar, and go in your go.mod. But it will not build or run any of them. To do that you have to run a command like go build or go test or go install. So you have the opportunity, between go get and the next go command, to run git diff and see what has changed in the go.mod. And while I haven't thought it through completely, it also seems like it is probably okay for go get foo to print a message about updating the go line, so that you don't even have to reach for git diff.

Another issue I have with automatic toolchain downloads is that this is another link in the security chain. ...

The toolchain downloads would be protected in exactly the same way as the module downloads. Personally I trust the computer to check checksums more than I trust myself to do it. If an attacker can substitute a module download without your noticing, then they might as well just insert code into the packages you are building and running. Being able to tamper with the toolchain as well doesn't substantially change what is possible.

[rsc]

Re: go.mod go line as a minimum version or something else

The design doc did not call enough attention to the implications of changing the go line to a minimum Go version, bringing it into line with all the other versions in go.mod that are minimum versions.

There are two questions: can we change it, and should we change it?

Can we change it? @zikaeroh pointed out a tweet about the go version being a “feature unlocker”. That's a reasonable description of what it is today, but not many people actually understand that. We hear from Go users all the time that they are confused about or misunderstand what it means and what it does and doesn't do. And obviously the people who reused it for version selection in Cloud Buildpacks and the setup-go GitHub Action seem to have misunderstood it as well. So I don't think we are locked in to the current semantics. If we can identify better semantics and document them clearly (and even better if they align with what people expect when they see first see that line), then we have plenty of room to get ourselves there. This is something we can change.

Should we change it? This is the more difficult question. I think we should, precisely because people are so often confused about what it means. If I see require 5.99.0 in a Perl script or edition = "2024" in a Cargo.toml, I know that older Perl or Cargo are simply going to reject that code. They're not going to try to compile it and hope for the best, like Go does. In fact I can't think of any other language that ignores this kind of version requirement. So why does Go? That's not rhetorical. It's worth understanding, in the Chesterton fence sense.

The original implementation of the go line (in CL 125940) did specify a minimum requirement. When we started enforcing that in the compiler, so that go 1.X modules didn't accidentally use features from Go 1.(X+1), @ianlancetaylor pointed out that people might get go 1.X in their module because that's what they were running, unnecessarily cutting off Go 1.(X-1) users from the module even if it would happen to build just fine with Go 1.(X-1). The most relevant comment is https://github.com/golang/go/issues/28221#issuecomment-432765035 (by me), specifically:

For a dependency asking for Go 1.(N+1) or later, when the current go version is Go 1.N, one option is to refuse to build. Another option is to try compiling as Go 1.N and see if the build succeeds. If we never redefine the meaning of existing syntax, then it should be safe to conclude from a successful build that everything is OK. And if the build fails, then it can give the error and say 'by the way, this was supposed to use a newer version of Go, so maybe that's the problem.' @ianlancetaylor has been advocating for this behavior, which would make as much code build as possible, even if people accidentally set the go version line too new. It sounds like this should work - in part because we've all agreed not to redefine the meaning of any existing syntax - so we should probably do it and only roll it back if there is a major problem.

That was in 2018. The costs and benefits of treating the go line as a strict minimum have shifted since then.

On the cost side, the only way in 2018 to implement the go line as a strict minimum was to refuse to build. That's a huge cost, and it would have forced lots more toolchain maintenance onto users. Most machines have a single toolchain installed globally for the entire machine, so updating it is a whole process and could break builds in unrelated projects. Or maybe you don't control the installed toolchain at all. Updating it is impossible. Or maybe it's just a big headache to deal with (see my earlier comment). Whatever the details, there's a high cost associated with having to switch toolchains, one that adopting “strict minimum” semantics would make users pay unnecessarily (at least in the cases where the listed version was not the true minimum version).

The proposal we are discussing here eliminates most of that cost. There would no longer be a single global toolchain for the machine, just like there's no longer a single global installed copy of any particular dependency package. The go command would manage multiple toolchains and select the right one for a given build the same way it does for modules. Now that its much easier to change toolchains, switching cost is no longer a reason to avoid strict minimum semantics.

On the benefits side, four years of not having strict minimum semantics have made clearer at least four benefits we are missing.

1. The first benefit is that the strict minimum semantics would be easier to understand. It's what users expect. That's why the tweet mentioned earlier said:

   @golang community. Reminder that the go directive in go.mod is a feature enabler for your current module and NOT a "minimum version of Go supported" indicator for your module's clients.

   We wouldn't have to write things like that, to tell people that their expectations are wrong, if the go line did what those people expect. (There would of course need to be clear documentation and communication about this meaning changing.)

2. The second benefit is that the strict minimum semantics provide the useful ability to say “you can't compile this code except with this version of Go or later.” The reported error can be made very clear, in contrast to the cryptic compile errors we get today (the design doc talks about these). If your program uses a new standard library package or function, it's much better for all your users if you can just write go 1.20 and rely on the toolchain to tell users they need to update, instead of making them first read errors about failed imports or unknown symbols and then mention the version mismatch.

3. The third benefit is that the strict minimum semantics let us make changes to what programs mean and signal very clearly to older Go toolchains that they should not try to compile these programs. When we did the //go:embed changes, we had to require writing import _ "embed" to use //go:embed on a string variable to break older toolchain builds. Otherwise, even though you have to write go 1.16 (or later) to use //go:embed, Go 1.15 toolchains would ignore the go 1.16, compile the code, miss the //go:embed comment, and write out a program without the embedded value. We made the experience worse (requiring the dummy import) because we didn't have strict minimum semantics.

   The same kind of thing can happen if you have written a package that only works properly with a bug fix included in a newer release. The best you can do to stop people from building with the older release is to insert an import or symbol reference to something unrelated that is new in that release. And there's nothing you can do if the bug is fixed in a point release. Later today we will release Go 1.19.4 with a bug that will keep you from using atomic.Pointer[T] to implement an atomic linked list. (Sorry. Wasn't caught in time and the train has left the station.) The compiler error if you try is inscrutable. That will get fixed in Go 1.19.5. With semantic minimums, if your code uses an atomic linked lists and therefore only works with Go 1.19.5 or later, you write go 1.19.5 in your go.mod and move on. Your users never see the weird errors that arise from Go 1.19.4.

   With things as they are today, there is nothing you can do to prevent those older builds and the confusion they will cause. This specific case of the atomic linked list is unlikely to arise, but similar things do happen. And there are bugs that don't result in compiler errors too, which is even worse for your users. Strict minimum semantics let you be sure no one is building your module with an incompatible toolchain and then decoding cryptic compiler errors or (worse) chasing down strange execution bugs that have already been fixed.

4. The fourth benefit is that strict minimum semantics let us fix semantic problems in the exceptional cases when that is appropriate. This will not arise often: perhaps just once. But that one is still compelling: we cannot safely fix loop scoping semantics unless we have some clear signal to older Go toolchains not to blindly compile code that depends on the newer semantics for correct execution. The most obvious, simplest, clearest such signal is to write go 1.99 (or whatever the right version is) in the go.mod file. If we don't have strict minimum semantics, I don't see how we fix loop scoping. I hope that loop scoping is the only time we will ever need this feature, but who knows? Perhaps another will arise after another 10 years of experience with Go. The quoted text above said "in part because we've all agreed not to redefine the meaning of any existing syntax". We've realized that was a mistake, at least in the exceptional case of loop scoping.

So compared to when we rolled back strict minimum semantics in 2018, we've found a way to remove most of the costs we were concerned about, and we now have a clearer understanding of which benefits we are missing out on. On balance, it seems to me that rolling strict minimum semantics forward again is a win.

There will be an effect on the ecosystem. We avoided strict minimum because we thought it would be too hard for users to upgrade. If we move to strict minimum because it is now easier for them to upgrade, yes, the effect will be quicker upgrades. I hope that most users won't think much about it at all. They'll just run 'go get newdependency', see the message about having a new Go version in this project, just like having a new indirect dependency, and continue on with their work.

@dominikh's comment was very insightful and is worth quoting in full:

I think the combination of auto-upgrading and not allowing old toolchains to build new code will change the Go ecosystem to either no longer support multiple versions of Go, or to delay experimenting with new language features.

Where previously a Go module could target Go 1.21 but make use of 1.22 features in optional, build-tagged files, it can no longer do so. To be allowed to use Go 1.22 language features, the go.mod has to specify go 1.22, but this triggers auto-upgrading / build failures on Go 1.21. The module now only supports Go 1.22.

Auto-uprading makes this seem fine at first: anyone who needs Go 1.22 will get it for free, automatically. However, I don't think auto-upgrading can be assumed to be pervasive. For one, most people who hack on Go at least occasionally will be using GOTOOLCHAIN=local, pointed at a git checkout. Furthermore, I predict that some Linux distributions will patch their Go packages to disable the automatic upgrading, as it side-steps their packages and some are allergic to that. This'll lead to users who are left behind even more than they're now due to slow-moving distributions.

Taking these paragraphs in reverse order, I think we can and should assume that Go toolchain management can be depended on and assumed to be pervasive. As I replied at length above, I think GOTOOLCHAIN=auto is perfectly compatible with Linux packager policies, and if packagers disagree then I'm happy to engage with them to understand that better and try to address their concerns. I expect that management of toolchain versions will become just as critical a part of Go as management of dependency versions, so we do need to make sure it's available to all our users. It is also important to emphasize (from above) that this proposal is not “auto-upgrading”, because there is no single pet toolchain to upgrade. (This is not like Chrome overwriting itself.)

It is true that a Go module will not be able to support Go 1.21 while also making limited use of Go 1.22 features using build tags. A different way to view this is that a Go module will not have to worry about that kind of complication anymore. If they want to use a Go 1.22 feature, they can easily use Go 1.22, confident that - due to the combination of this proposal and #56986 - users who want the latest version of the module can use Go 1.22 with that project instead of being “stuck” on Go 1.21. Yes, the module now only supports Go 1.22, just like the current Kubernetes only supports github.com/emicklei/go-restful/v3@v3.9.0. Users who want the latest of the module use the updated dependencies it requires.

I don't think it's strictly accurate to say the Go ecosystem wouldn't support multiple versions of Go. That's like saying the Go ecosystem doesn't support multiple versions of go-restful or go-yaml. Different modules will make different decisions about whether to start using the latest Go version. I do think it is accurate to say that it will not be as important for popular Go ecosystem packages to keep their latest module versions working with the last two Go toolchain versions, and that perhaps as a result more will choose to move forward more quickly than in the past. That's fine with me. The main reason these popular packages needed to support older Go versions was for people with environments where it was too costly or impossible to move to a newer Go version. This proposal and #56986 aim to remove those costs and impossibilities, which will make it less important for these packages to support older Go versions. I am sure better things can be done with the time currently dedicated to that support.

Overall, the comment is spot-on. We are explicitly aiming to remove barriers to updating to the newest Go version. That should in turn let the entire Go ecosystem adopt newer versions more quickly, which should be a win for everyone.

[rsc]

@rittneje

One final feature of treating the go version this way is that it would provide a way to fix for loop scoping, as discussed in discussion #56010. If we make that change, older Go toolchains must not assume that they can compile newer Go code successfully just because there are no compiler errors.

Existing versions of the Go compiler already do not fail just because the go directive is newer.

Sorry for the delay. It took me a while to realize what you were getting at. You are absolutely right that existing Go versions do not reject code with newer go lines. That's why we can't do the for loop scoping immediately. Instead, the best plan I have is to land the new go line interpretation in Go 1.X and then wait to land loop scoping until Go 1.(X+1). When Go 1.(X+1) comes out, Go 1.(X-1) and earlier will be officially unsupported, and Go 1.X will know enough to refuse to build Go 1.(X+1) code. So no supported Go version will miscompile new loop code.

[willfaught]

We have to make it easy for people to understand what toolchain they are using and why. The way to do that has not changed: run go version and it will tell you.

@rsc If go version now reports the toolchain version that would be used for the current module, is there now no way to tell from the command line what the bundled Go version is?

[Foxboron]

Quick point while I contemplate if it's worth engaging on this topic as the Arch maintainer for the go package.

One option is to trust in Go's high-fidelity, reproducible builds and let the go command fetch the dependencies directly. I would hope that systems that take this approach are also comfortable letting the go command fetch any toolchain dependency as well, since the toolchain fetches have the same high-fidelity, reproducible behavior as module dependency fetches.

It's a very big difference between downloading sources files defined in the go.mod files and fetching binary files files from some remote location. We are all very aware of the trusting trust attack and moving the reproducible builds requirements from the downstream distributor (Linux distributions) to the upstream (Google) is not trivial.

So how is Google going to provide Reproducible Builds for the downloaded toolchains?

[rsc]

@willfaught GOTOOLCHAIN=local go version would do it.

[rsc]

@Foxboron, regarding "Reproducible Builds", by that do you mean https://reproducible-builds.org/? And if so what is involved in "providing" one? As of Go 1.21 we expect our toolchains will be fully reproducible even when cross-compiling. (That is, if you build a Mac toolchain on Windows, Linux, and Mac, you get the same bits out in all cases.) I would be delighted to have a non-Google project reproducing our builds in some way.

[Foxboron]

regarding "Reproducible Builds", by that do you mean https://reproducible-builds.org/?

Yes. I have been working on this project since 2017 for Arch Linux.

And if so what is involved in "providing" one?

If this gets implemented we would be downloading binary toolchains, right? I want to reproduce the binaries distributed by Google.

Just checking out the source and building versions won't necessarily be enough, so there needs to be some attestation or SBOMs published to support the distribution of the binaries.

I'm not saying this can't be done. I'm just trying to point how the bar between the "reproducible builds" Go already facilitates with source code is very different from what you would need to ensure for binary builds.

I would be delighted to have a non-Google project reproducing our builds in some way.

I'm not sure if "our builds" is the distributed binaries from Google? But Arch has been publishing verifiable builds of the Go compiler for 2 or 3 years now.

[rsc]

@Foxboron, I copied these messages to #57120 and will reply there.