Open mholt opened 5 years ago
go get
a Go project's source code and install its command (the repo contains both a command and library packages) while respecting module dependencies.
Note that GO111MODULE=on go get
does this today. It just unpacks the source code into the module cache instead of GOPATH/src
.
There is currently some question as to what to do about replace
directives (see #31173), but if you don't need those then you should be set as far as installing binaries.
But setting
GO111MODULE=on
also doesn't work because the beta version isn't recognized. The last non-prerelease tag doesn't use modules, so the build also fails.
Yep, that behavior is expected and intended. We want the choice to use an unreleased version (when released alternatives exist) to be explicit.
You can always go get […]@master
or go get […]@v1.0.0-beta1
. (Or finish up that release and tag it! 🙂)
what is the alternative for "developers" as opposed to just "builders"?
Generally we're trying to move the go
command away from autonomously invoking VCS tools in potentially-nested directories, since that has historically been a source of security vulnerabilities.
The alternative for developers is to run git clone
(or your alternative VCS of choice) explicitly. If that alternative doesn't work for you, please let us know more detail so that we can understand the problem and the use-cases.
(See also #18387.)
(CC @jayconrod, but I don't think we'll be able to make significant changes here until 1.14 at the earliest.)
Thanks @bcmills for taking a quick look at this.
(Sorry for the close+open earlier; my trackpad sometimes does phantom clicks.)
I had always viewed go get
as a tool to "go get" source code and populate my GOPATH so that I can work on it.
But now with GO111MODULE=on, and with Go 1.13, I will need to view it almost solely as a dependency manager. (This is a significant change that I think is not obvious.)
I agree it seems like a good idea to not get pre-releases by default... just makes the transition awkward since our beta release is the first to try using modules.
Running git clone
is reasonable I suppose, just something new to my workflow which I never had to do much with Go code.
@thepudds gave this 5-star explanation in Slack:
The way it works is if you are inside of a module
foo
and dogo get some/cmd
, thensome/cmd
is treated as a dependency of your current modulefoo
, complete with resolving any shared version constraints with the versions listed ingo.mod
forfoo
, updating thego.mod
forfoo
to include arequire
forsome/cmd
, and storing the code forsome/cmd
in the module cache just like other dependencies offoo
.The way it works if you are outside of a module with GO111MODULE=on is briefly documented here:
https://golang.org/doc/go1.12#modules
When GO111MODULE is set to on, the go command now supports module-aware operations outside of a module directory, provided that those operations do not need to resolve import paths relative to the current directory or explicitly edit the go.mod file. Commands such as go get, go list, and go mod download behave as if in a module with initially-empty requirements. In this mode, go env GOMOD reports the system's null device (/dev/null or NUL).
In other words, in that case, if you do
cd /tmp; go get some/cmd
, thensome/cmd
is still treated "as if"some/cmd
is a dependency of a temporary and otherwise empty module, complete with storing the code forsome/cmd
in the modules cache along with dependencies of other modules.
So now I think I have something I can tell the community, many of whom are currently having trouble building Caddy from source:
git clone https://github.com/mholt/caddy.git
cd caddy
git checkout <version>
:arrow_left: optional, but necessary if wanting a specific versiongo build
or go install
:arrow_left: will get all dependencies as specified by go.mod
Thank you for the help, coming to understand this.
I'll leave the issue open since being able to consolidate those steps into one command (like it has always been before) would be convenient, especially if one (like me! :smile:) wants to continue to use a traditional GOPATH. In other words, being able to go get
source code as before, but with respect to go.mod
, would be great.
One last question: in the future, GOPATH is still where module dependencies will be stored (read-only); will it still be recommended and good practice to store working copies of repos in GOPATH? I ask because I really like its structure, but if it messes with go tooling + modules, I should try to avoid that...
One last question: in the future, GOPATH is still where module dependencies will be stored (read-only); will it still be recommended and good practice to store working copies of repos in GOPATH? I ask because I really like its structure, but if it messes with go tooling + modules, I should try to avoid that...
$GOPATH/src
is still a good place to keep working copies of repositories. I'd recommend it to keep code organized. However, since the Go command won't look for dependencies there, avoid writing tools or scripts that require this convention.
@bcmills can you explain in more details why was this behavior changed?
I really liked the simple instructions to build a binary
go get github.com/mholt/caddy
cd $GOPATH/src/github.com/mholt/caddy
go build cmd/caddy
didn't need to think what is my PWD and didn't need to create all parent folders etc.
@krasi-georgiev @bcmills in addition to downloading source code (which has been discussed here already), go get
of a main
package would by default install to $GOPATH/bin
, so instructions for someone to download and run your tool were simply:
$ go get github.com/foo/cmd/bar
$ $GOPATH/bin/bar
Or if $GOPATH/bin
is in one's $PATH
, simply
$ go get github.com/foo/cmd/bar
$ bar
In Go 1.13 I really miss this functionality. Now I can't do this, or tell someone else to do it, without likely damaging some unrelated go.mod
file.
In other words, we used to have a great tool for downloading and installing source code and binaries, now we have a tool that might do that in a sane way depending on the context, or it might instead update module dependency requirements (and install the binary to $GOPATH/bin
, which doesn't seem to be the right thing to do in this context).
It would be great if these two very separate use cases were not both crammed into the same command go get
.
For example a quick twitter search shows lots of examples of people giving instructions to install their tool via go get
: https://twitter.com/search?q=%22go%20get%22%20%23golang&src=typed_query&f=live
I don't think updating a go.mod
file is what is intended in any of those cases. Asking people to set GO111MODULE
a certain way to get the right behavior feels like a kludge.
@jnjackins #30515 is the issue for that.
Waiting on this. Having to specifically clone all of my projects to the right location in the tree in order to have the replaces all make sense in all actively developed modules just adds extra setup/accounting to development. I appreciate modules in all of their advantages, but it's backwards that this is no longer an automatic part of the workflow. Before, everything was self organizing.
This somewhat reminds me of "automatically"- versus "manually"-installed packages in Ubuntu. The sourcecode for the project(s) passed as arguments to go-get will be brought down as source under $GOPATH/src though their dependencies (if a module) will be established under pkg/. If the dependencies are not modules then just bring those down as source as well. This should maintain the existing ethos of which stuff should be under pkg/ and which should be under src/. This probably oversimplifies the requirements and their ramifications, but any progress in this direction would be good.
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat are you trying to do?
(Added this question, hope that's OK.)
go get
a Go project's source code and install its command (the repo contains both a command and library packages) while respecting module dependencies.In the past we used
vendor
to pin dependencies, but we'd rather use Go modules.go get
has always worked great, even with an empty GOPATH. If possible, we'd like for the project to stay go-gettable.What did you do?
export GOPATH=
and ensure no~/go
folder exists)$ GO111MODULE=on go get github.com/mholt/testrepo/caddy
What did you expect to see?
The
$GOPATH/src
(~/go/src
) folder populated with the project's source code, and the binary installed to thebin
folder, while honoring the versions specified in go.mod.What did you see instead?
Only module folders were created.
How does one get the source code, with dependencies, while honoring
go.mod
?The one-step install flow of
go get
was really nice. Will this no longer be the case with Go 1.13 whenGO111MODULE=on
becomes the default behavior?More info
We're in an awkward position in our transition to modules. Many people are unable to build Caddy from source without some wrangling.
We've added go.mod and go.sum, removed our vendor folder, and tagged a beta release (in the actual repo, not my test repo here). Unfortunately, modules seem to ignore the beta release tag so all module operations fail unless we explicitly specify the beta version in our commands. So it is not obvious how to test that we've transitioned correctly.
Additionally, without modules (GO111MODULE=off), running
go get
on Caddy currently fails with:because one of our upstream dependencies have had a breaking change. Our go.mod pins the working version of this dependency, but
go get
does not honor this (yet?).But setting
GO111MODULE=on
also doesn't work because the beta version isn't recognized. The last non-prerelease tag doesn't use modules, so the build also fails.(As an aside,
@v1.0.0-beta1
needs to be specified explicitly, which (temporarily?) breaks our build instructions and isn't obvious... it confused a lot of people including me.)Even still, if we start over on the test repo but set
GO111MODULE=on
this time, it does not pull down the source code so we can start developing on it.If
go get
is how the GOPATH is populated for development, it should put the source code in it so we can develop with it. Ifgo get
becomes a command for a read-only workflow, what is the alternative for "developers" as opposed to just "builders"?