search

golang / dep

Go dependency management tool experiment (deprecated)
https://golang.github.io/dep/
BSD 3-Clause "New" or "Revised" License
12.85k stars 1.05k forks source link

dep ensure -v hangs forever #1778

Closed alacret closed 4 years ago

alacret commented 6 years ago

dep ensure -v hangs forever with dependencies that do not exist.

For example:

I'm refactoring a package, that i have under github.com/my-username/my-package, so i create a new folder: github.com/my-username/my-new-refactor-package, i haven't push any code to that repo neither create it, so the commands hang try to solve the dependency.

It would be good if it have a timeout or a check to this.

alacret commented 6 years ago

I remove the depencies to the extornal unpushed package and still hangs:

(i also remove the descriptors to try to start again)

dep init -v 13:09:41 Getting direct dependencies... Checked 1 directories for packages. Found 2 direct dependencies. Root project is "github.com/4geeks/pex" 1 transitively valid internal packages 3 external packages imported from 2 projects (0) ✓ select (root) (1) ? attempt github.com/mattn/echo-livereload with 1 pkgs; 1 versions to try (1) try github.com/mattn/echo-livereload@master

when i hit Ctrl + C

dep init -v 13:09:41 Getting direct dependencies... Checked 1 directories for packages. Found 2 direct dependencies. Root project is "github.com/4geeks/pex" 1 transitively valid internal packages 3 external packages imported from 2 projects (0) ✓ select (root) (1) ? attempt github.com/mattn/echo-livereload with 1 pkgs; 1 versions to try (1) try github.com/mattn/echo-livereload@master ^CSignal received: waiting for 1 ops to complete... (1) ✓ select github.com/mattn/echo-livereload@master w/1 pkgs (2) ← no more versions of github.com/concourse/concourse to try; begin backtrack ✗ solving failed

Solver wall times by segment: b-list-versions: 10m7.871632321s b-list-pkgs: 262.714406ms b-gmal: 244.690055ms select-atom: 236.542µs select-root: 218.67µs satisfy: 130.388µs new-atom: 116.779µs b-deduce-proj-root: 106.561µs other: 47.291µs unselect: 41.21µs backtrack: 6.341µs b-source-exists: 4.316µs

TOTAL: 10m8.37994488s

alacret commented 6 years ago

I try -golang because my dependency is already in my GOPATH, but still hangs

dep init -v -gopath (1) 13:34:24 Getting direct dependencies... Checked 1 directories for packages. Found 2 direct dependencies. Searching GOPATH for projects...

The following dependencies were not found in GOPATH: github.com/labstack/echo github.com/mattn/echo-livereload

The most recent version of these projects will be used.

Root project is "github.com/4geeks/pex" 1 transitively valid internal packages 3 external packages imported from 2 projects (0) ✓ select (root) (1) ? attempt github.com/mattn/echo-livereload with 1 pkgs; 1 versions to try (1) try github.com/mattn/echo-livereload@master ^CSignal received: waiting for 1 ops to complete... (1) ✓ select github.com/mattn/echo-livereload@master w/1 pkgs (2) ← no more versions of github.com/concourse/concourse to try; begin backtrack ✗ solving failed

Solver wall times by segment: b-list-versions: 8m2.037396579s b-list-pkgs: 267.649871ms b-gmal: 240.530492ms select-atom: 257.036µs satisfy: 177.31µs select-root: 163.58µs b-deduce-proj-root: 149.831µs new-atom: 108.428µs other: 31.358µs unselect: 21.157µs backtrack: 4.368µs b-source-exists: 3.935µs

TOTAL: 8m2.546493945s

alacret commented 6 years ago

This just does not work, maybe it's . version problem?

` dep ensure -v (130) (*master+1) 19:22:38 Warning: the following project(s) have [[constraint]] stanzas in Gopkg.toml:

✗ github.com/mattn/echo-livereload

However, these projects are not direct dependencies of the current project: they are not imported in any .go files, nor are they in the 'required' list in Gopkg.toml. Dep only applies [[constraint]] rules to direct dependencies, so these rules will have no effect.

Either import/require packages from these projects so that they become direct dependencies, or convert each [[constraint]] to an [[override]] to enforce rules on these projects, if they happen to be transitive dependencies,

Root project is "github.com/4geeks/pex-product-api" 1 transitively valid internal packages 3 external packages imported from 2 projects (0) ✓ select (root) (1) ? attempt github.com/labstack/echo with 2 pkgs; at least 1 versions to try (1) try github.com/labstack/echo@3.3.4 (1) ✓ select github.com/labstack/echo@3.3.4 w/2 pkgs (2) ? attempt github.com/dgrijalva/jwt-go with 1 pkgs; at least 1 versions to try (2) try github.com/dgrijalva/jwt-go@v3.2.0 (2) ✓ select github.com/dgrijalva/jwt-go@v3.2.0 w/1 pkgs (3) ? attempt github.com/valyala/fasttemplate with 1 pkgs; at least 1 versions to try (3) try github.com/valyala/fasttemplate@master (3) ✓ select github.com/valyala/fasttemplate@master w/1 pkgs (4) ? attempt github.com/labstack/gommon with 4 pkgs; at least 1 versions to try (4) try github.com/labstack/gommon@0.2.4 (4) ✓ select github.com/labstack/gommon@0.2.4 w/4 pkgs (5) ? attempt github.com/mattn/go-colorable with 1 pkgs; at least 1 versions to try (5) try github.com/mattn/go-colorable@v0.0.9 (5) ✓ select github.com/mattn/go-colorable@v0.0.9 w/1 pkgs (6) ? attempt github.com/mattn/go-isatty with 1 pkgs; at least 1 versions to try (6) try github.com/mattn/go-isatty@v0.0.3 (6) ✓ select github.com/mattn/go-isatty@v0.0.3 w/1 pkgs ^CSignal received: waiting for 1 ops to complete... (7) ← no more versions of github.com/4geeks/pex-amazon-api to try; begin backtrack ✗ solving failed

Solver wall times by segment: b-source-exists: 6h0m26.164744209s b-list-pkgs: 1.602364396s b-gmal: 1.504512501s b-deduce-proj-root: 370.166805ms select-atom: 906.391µs satisfy: 862.255µs new-atom: 590.107µs select-root: 135.206µs other: 85.514µs unselect: 50.848µs backtrack: 4.236µs

TOTAL: 6h0m29.644422468s

`

pierrebiver commented 6 years ago

I have the exact same problem. Can someone point out a solution, please?

xjtian commented 6 years ago

This has happened to me before as well, when I accidentally imported from a private repository.

pierrebiver commented 6 years ago

I fixed it by having the same path from github to my local environment with all my project. in you src folder, if you have a a github.com folder, check that all your project there match a github url, exemple: github repo link -> https://github.com/pierrebiver/myproject
local repo folder $GOPATH/src/github.com/pierrebiver/myproject Otherwise dep stay stuck and cannot resolve your project with an exisiting github repository. Would be nice to have a timeout or that dep skip repo it cannot resolve with some warnings.

otterley commented 6 years ago

Another reason why this might fail is if you're running dep in a container that does not have access to Github. I was able to address this problem by mounting my $HOME/.netrc file into the container's user's home directory.

rodrigo-brito commented 6 years ago

Solved! Same problem here. Dep has a problem with private repositories, if you use your SSH key for fetch, use the command: ssh-add -K ~/.ssh/id_rsa to add your key to the keychain.

dryaf commented 6 years ago

I did delete a github repo which was required and also got the hang the repo wasn't mentioned in any Gopkg file, just in one test.

jkrnak commented 6 years ago

It is happening with private repos and non existing repos. A timeout would be nice. If you are experiencing this in a container. You can work around inside a container by mounting up your ~/.ssh to /root/.ssh and then starting and ssh agent and adding your key. Note: your ssh key name might be different. 

eval "$(ssh-agent -s)"; ssh-add /root/.ssh/id_rsa
jkrnak commented 6 years ago

Or... https://golang.github.io/dep/docs/FAQ.html#how-do-i-get-dep-to-consume-private-git-repos-using-a-github-token set up .netrc with github token and mount the file (still talking about containers)

anguianoa commented 6 years ago

I configure my system with .netrc and dep still hangs. Any other suggestions? I'm thinking of setting up ssh to github. I would think that using dep on your own private repositories would work out of the box. I'm not able to find a tutorial or example on the internet. (this is maybe the wrong thread, my issue is that dep isn't working at all for me it hangs.)

anguianoa commented 6 years ago

For the record, I found why my system hung. It was working, using the .netrc file. My issue was that in my repository it had bad imports that reference nonexisting packages. Dep hangs forever with no clues. The timeout of resolving the bad import and the name of the import that timed it out would have been great. I"m new to dep, so I spent a lot of time figuring this out. I hope if someone else runs into this they will find this comment.

EugenMayer commented 6 years ago

I have the exact same issue when buildin in CI. I am aware of some quirks described here.

I am building the same project locally and it works without issues. When i build in the CI, i use this image to build in: https://github.com/EugenMayer/docker-image-golang-builder. You can use it by docker run --rm -it -v yourgoporject:/src eugenmayer/golang-builder bash

The project:

deb ensure works locally with the same git url override / config

So i really picked up as much as i can find to fix this, still, in that container, which is just a usual golang official image i get:

dep ensure -v
(1/33) Wrote github.com/docker/docker-credential-helpers@v0.6.1
(2/33) Wrote golang.org/x/sys@master
(3/33) Wrote github.com/Nvveen/Gotty@master
(4/33) Wrote github.com/docker/go-connections@v0.3.0
(5/33) Wrote github.com/gogo/protobuf@v0.4

There are a gazzilion of issues here, being a duplicate to that:

I think we finally need tooling to find out what is causing this - this is a deal breaker for me and i am about to migrate back to glide due to this - but i would love to not do:

We need to find out which of those issues above are: a) Stuck due to network related issues: ssh transport issues, github token issue and dep needs to finally make them visible. Why are there no timeouts? Why are responses of the ssh access client not parsed to the cli (e.g. if interactive is required?) b) which are due to version lookup issues, cyclic project includes or whatever c) ? Things i do not know about?

Please help us with a dep tool exposing far more informations, -d for debug would be what we need here. @sdboyer please be a hero :)

EugenMayer commented 6 years ago

corrected I tried different dep / golang versions

twiest commented 6 years ago

I'd like to add another one to @EugenMayer's list.

d) external software dependencies (e.g. mercurial) are checked and an error is issued if not found.

I just had an issue where dep was hanging because merucrial wasn't installed. Once I installed it, it worked perfectly. It (of course) took several hours to figure out that was the issue. :)

mvdan commented 4 years ago

Dep was officially deprecated earlier this year, and the proposal to archive this repository was accepted. As such, I'm closing outstanding issues before archiving the repository. For any further comments, please use the proposal thread on the Go issue tracker. Thanks!