Closed dzrw closed 7 years ago
@politician Welcome to dep
! Thanks for the detailed issue.
I assume you are trying to compile dep
? You don't need to get the dependencies of dep
using dep ensure
since our vendor directory is already committed. Just doing go get -u github.com/golang/dep/cmd/dep
is enough to compile a binary in GOPATH/bin
.
At the moment, you can't run dep
in a project that doesn't exist within a GOPATH/src
subdirectory.
So assuming you want to try dep ensure
in dep
itself, you will need to ensure that you clone it in GOPATH/src/github.com/golang/dep
(which go get
most likely did).
Both /Users/politician/code/golang/dep
and /Users/politician/Documents/Code/golang/dep
are not within your GOPATH
(which appears to be /Users/politician/.go/workspace
. Setting the GOPATH
to PWD
also doesn't solve the issue since you code now is at the GOPATH
root.
@ibrasho hahaha, you commented again just while I was drafting mine π Anyway, since I have drafted it, I'm posting it. Just my attempt to help.
Hi and welcome,
Thanks for trying dep. I'll share experiences from my workflow and try to help you out π
To get the code, I use go get -u
and it clones the project to my default GOPATH workspace. Once I have the code, I use this same repo to develop dep. Since, it's already under a GOPATH and properly cloned, I don't have to worry about anything else.
Neither GOPATH nor GOROOT are set. I fetch dep per the README. It lands in a go workspace that I mostly ignore and never work out of.
I believe you have to work under a GOPATH, that's how all go projects are supposed to work at the moment. And you have to set a GOPATH. You can refer The GOPATH environment variable to set your GOPATH env var.
I see your go workspace is under /Users/politician/.go/
, which isn't the default path if GOPATH is set, I guess. So ensure that your GOPATH is set to /Users/politician/.go/workspace
.
dep ensure
in dep itself works fine if the above setup is done right. But you don't have to run ensure
because all the dep dependencies/vendors are already committed. Just fetching the code and running go install -v ./cmd/dep/
from the dep project root, should build and install dep from your copy of source.
And if you wanna contribute to dep with the same copy of repo, I would recommend reading GitHub and Go. It talks about setting up another git remote and all the things that you would need to contribute to any go project.
Finally, since this repo includes a Gopkg.toml, it has a multi-step build process -- at least dep ensure and go build .... Would consider including a Makefile to make that build process explicit for newcomers?
We don't run ensure
anytime, until we decide to add another dependency. But since the dependencies are committed under vendor/
, there's no need to ensure
. And hence we don't need Makefile right now, but we might have one in future, if required.
Hope that was helpful.
@ibrasho @darkowlzz Thanks for the quick response! I think I'm beginning to understand what's gone wrong.
@ibrasho said,
At the moment, you can't run dep in a project that doesn't exist within a GOPATH/src subdirectory. So assuming you want to try dep ensure in dep itself, you will need to ensure that you clone it in GOPATH/src/github.com/golang/dep (which go get most likely did).
Ok, I've found line 247 in context.go that makes the assumption that GOPATH is the default location referred to in the Go documentation. The way I use Go is different from the Go documentation because I work on many projects written in different languages on a daily basis.
Therefore my project area puts all of these projects side-by-side and mirrors the way Github, Bitbucket, and Gitlab structure their repository layout. For example,
~/code/golang/dep
contains my dep
clone~/code/politician/juypter-notebooks
contains my jupyter python notebooks~/code/miAmidos/dcind
contains a Docker-in-Docker container exampleVery different code bases, but the project paths all follow a similar path structure which makes it easy for me to navigate. Using gb
allows me to maintain that same path structure for Go projects, but unfortunately, it looks like dep
would require any dep-based project to be cloned into either:
clone-into | set GOPATH to | example path |
---|---|---|
~/.go/workspace/src |
~/.go/workspace |
~/.go/workspace/src/github.com/golang/dep |
~/code/dep-projects/src |
~/code/dep-projects |
~/code/dep-projects/src/github.com/golang/dep |
GOPATH/src | (whatever the default Go install GOPATH is) | $GOPATH/src/github.com/golang/dep |
Takeaway 1: There is a strong assumption in dep
that Go source code is located an folder structure that closely resembles the Go monorepo workspace concept. In particular, it is not possible to use a folder structure with a different top-level structure; dep
does not orient itself around Gopkg.toml
.
@darkowlzz said,
I believe you have to work under a GOPATH, that's how all go projects are supposed to work at the moment. And you have to set a GOPATH. You can refer The GOPATH environment variable to set your GOPATH env var.
Interesting, so it turns out that when using gb
you don't have to set a GOPATH because the tool manages it for you. I believe that read something about Go getting rid of the GOPATH around the same time that dep
started up, so since I haven't been using GOPATH explicitly for years it looks like what's happened is that I've forgotten what other folks deal with.
Now, I do need to set GOPATH to allow go get
to know where to install binaries, so on the rare occasions that I go get
something, I'll set the GOPATH to ~/.go/workspace
.
From your description, it almost sounds like most folks are working out of a structure that also contains random 3rd party binaries and packages, and that they use another folder structure for handling projects written in other languages. Wild!
@darkowlzz said,
We don't run ensure anytime, until we decide to add another dependency. But since the dependencies are committed under vendor/, there's no need to ensure.
Ok, awesome. That makes sense now. I didn't realize that there was no need to fetch the vendored dependencies after cloning because I mistakenly thought that the presence of the Gopkg.toml
and Gopkg.lock
files implied a different workflow (i.e. the nonreproducible builds / npm / bundler workflows).
Takeaway 2: Don't automatically assume that you need to call dep ensure
upon cloning a repo containing a Gopkg.toml
file. Look for the vendor/
directory first, if it's there, then you're good to go.
Would you guys be interested in entertaining a patch that allowed my use case to work?
I haven't thought deeply about it. Perhaps one could enable dep to discover Gopkg.toml
and add it to the GOPATH in a subshell, or perhaps one could add a command-line flag that overrides the default behavior in some way.
dep
is being developed with the goal of being absorbed into the go
toolchain. So I'm not sure we want to support cases that don't comply with the toolchain assumptions. The final call of this is left to @sdboyer though. π
It seems gb
is a totally different way for working with Go. I can propose the following workaround to keep you preferred structure and still be able to use dep
.
You can either set GOPATH to ~/.go/workspace
or move it to ~/go
(the default GOPATH
in Go 1.8). If you move it, everything in the go toolchain should know your GOPATH even if it is not set.
Use GOPATH/src
to store the repositories and use symlinks to keep your current structure. Thus, /Users/politician/code/golang/dep
(or /Users/politician/Documents/Code/golang/dep
) should be a symlink to GOPATH/src/github.com/golang/dep
.
This should work for dep
, but I'm still not sure how it'll impact gb
behavior.
Thank you for that clarification and the workaround.
It seems gb is a totally different way for working with Go.
I wouldn't put this all on gb
, for all I know my team and I are the only ones writing code in this way. π
hi @politician, thanks for such a patient and detailed writeup!
Being able to work from anywhere, not just within GOPATH, is certainly something we'd like to support. But it has deep implications and, as @ibrasho notes, takes us out of alignment with the go toolchain in a way that we can't currently afford to do. We do have some plans about how we might resolve it eventually, but unfortunately, there's little we can do safely right now. π’
I'm having trouble building
dep
locally on my MacBook Pro. Below is a trace of what I've tried, some facts about my environment, and the errors raised. In particular, I cannot get past downloading dep's dependencies usingdep
. Hopefully, someone can point out what's wrong with my setup.I'm running Go 1.8. Neither GOPATH nor GOROOT are set. I fetch
dep
per the README. It lands in a go workspace that I mostly ignore and never work out of.I clone the repo into my project area, and guess that I need to run
dep ensure
because I see that you have a Gopkg.toml file. This fails. I did not expect this error because elsewhere I've read thatdep
considers theGopkg.toml
to define the project root, so I expected it to be able to self-orient.Note that ~/code is a symlink to ~/Documents/Code. This may or may not be relevant.
I set $GOPATH to the current directory and try again. No dice.
Discussion
dep
fails in this exact same way for a couple other projects. I try to pull them down, and rundep ensure
and it fails. I've tried several combinations of setting GOPATH to various different places and trying multi-entry GOPATHS. I've looked for, but haven't found any documentation on how to setup the environment to be compatible.Now, I've been a heavy user of Dave Cheney's
gb
for several years. However, sincedep
seems like it's going to be the official way forward at some point, it seemed like a good idea to get some experience with it. Unfortunately, it's likely that I've forgotten something fundamental about how to usego build
directly sincegb
takes care of most of the annoying bits.Other folks seem to be able to use
dep
successfully, including some of my colleagues, but no one can explain why it doesn't work on my machine. I hope I've provided enough detail to figure out what's different about my environment, but let me know if there's anything I can provide to troubleshoot this.Finally, since this repo includes a Gopkg.toml, it has a multi-step build process -- at least
dep ensure
andgo build ...
. Would consider including aMakefile
to make that build process explicit for newcomers?