dep ensure running for ever

[piedup]

Hi

I am trying to install gibson042/canonicaljson-go but the dep ensure command seems to run forever. If you kill it (Ctrl+C) and run it again it works. I've tried forking the repo and removing .gitmodules and it works too.

When trying on my Mac dep ensure work fine.

I have created a repo where you can reproduce the issue using a Dockerfile: piedup/canonical-test

The fact that it works after killing it seems odd.

go version

go version go1.9.2 linux/amd64

What version of dep are you using (dep version)?

dep: version : devel build date : git hash : go version : go1.9.2 go compiler : gc platform : linux/amd64

What dep command did you run?

dep ensure

What did you expect to see?

vendor directory getting published

What did you see instead?

dep ensure keeps running endlessly

[ibrasho]

I'm not able to replicate this on my Mac but I can confirm that the container hangs for a long time.

[ibrasho]

The container has git version 2.11.0 while my machine has git version 2.14.1. Not sure if this is relevant.

What's the git version on your machine?

[piedup]

Running on my Mac with git 2.13.5 works as well. I think it is more a linux issue. It also fails on linux using git version 2.14.1.

[piedup]

@ibrasho is someone looking into this? We're using docker containers to deploy our code so having dep working on linux is quite critical.

Do you need anything from my end?

[sdboyer]

so having dep working on linux is quite critical.

please understand that dep is an open source project, and we are all volunteers - critical for you is not necessarily the same as critical for the project, and we have to ration the limited time we have available on the basis of the latter, not the former.

Do you need anything from my end?

while we have, unfortunately, limited visibility right now into where exactly dep is hanging, you might be able to get a better sense of it by running dep ensure -v and reporting at what point in the output the process seems to hang.

[piedup]

Of course I understand. I just wanted to make sure that you had everything you needed when you have a chance to look into this.

I have tried dep ensure -v but it hangs directly and doesn't log anything.

[sdboyer]

Of course I understand.

thanks!

I have tried dep ensure -v but it hangs directly and doesn't log anything.

ok, that's most likely stuck on initial cloning, then.

my first hunch would be that it's got something to do with docker not liking the use of the ssh URL for the gitmodule. not sure exactly where that breakdown would be occurring, though.

i've got to switch to other things today, but you could help out by validating this hypothesis quickly - clone down the latest dep, then delete this line: https://github.com/golang/dep/blob/832c144821d691b7221ea2b9185473d073526a4d/gps/vcs_repo.go#L89, and use that version of dep, and see if it still hangs.

[piedup]

Ok thank you I will try this.

[dpinela]

I was having this same issue on freebsd/arm64. I tried removing the --recursive option from the git call as @sdboyer suggested and that fixed it.

[piedup]

I can confirm that by removing --recursive, dep ensure works just fine. I am not sure what the long term fix should be.

Also I tried forking the dependence (gibson042/canonicaljson-go in that case) and removed the .gitmodules file, dep ensure also works.

What I am struggling to understand is why it works on Mac but not on Linux.

[sdboyer]

well i'm glad we've isolated the failure, at least 😄

if you manually git clone --recursive that repository, do you see the same behaviors? (both @dpinela and @piedup)

[piedup]

On my computer (Mac) it works:

git clone --recursive http://github.com/gibson042/canonicaljson-go
Cloning into 'canonicaljson-go'...
warning: redirecting to https://github.com/gibson042/canonicaljson-go/
remote: Counting objects: 189, done.
remote: Compressing objects: 100% (66/66), done.
remote: Total 189 (delta 117), reused 189 (delta 117), pack-reused 0
Receiving objects: 100% (189/189), 92.11 KiB | 230.00 KiB/s, done.
Resolving deltas: 100% (117/117), done.
Submodule 'canonicaljson-spec' (git@github.com:gibson042/canonicaljson-spec.git) registered for path 'canonicaljson-spec'
Cloning into '/Users/pierre/Documents/canonicaljson-go/canonicaljson-spec'...
remote: Counting objects: 293, done.        
remote: Total 293 (delta 0), reused 0 (delta 0), pack-reused 293        
Receiving objects: 100% (293/293), 45.51 KiB | 163.00 KiB/s, done.
Resolving deltas: 100% (101/101), done.
Submodule path 'canonicaljson-spec': checked out '59f96a13322d8148a95f54d0e06ca48f7a6e4dd3'

But it fails within on linux (within a docker container):

git clone --recursive http://github.com/gibson042/canonicaljson-go
Cloning into 'canonicaljson-go'...
remote: Counting objects: 189, done.
remote: Compressing objects: 100% (66/66), done.
remote: Total 189 (delta 117), reused 189 (delta 117), pack-reused 0
Receiving objects: 100% (189/189), 92.11 KiB | 0 bytes/s, done.
Resolving deltas: 100% (117/117), done.
Submodule 'canonicaljson-spec' (git@github.com:gibson042/canonicaljson-spec.git) registered for path 'canonicaljson-spec'
Cloning into '/go/src/github.com/piedup/canonical-test/canonicaljson-go/canonicaljson-spec'...
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights and the repository exists.
fatal: clone of 'git@github.com:gibson042/canonicaljson-spec.git' into submodule path '/go/src/github.com/piedup/canonical-test/canonicaljson-go/canonicaljson-spec' failed
Failed to clone 'canonicaljson-spec'. Retry scheduled
Cloning into '/go/src/github.com/piedup/canonical-test/canonicaljson-go/canonicaljson-spec'...
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights and the repository exists.
fatal: clone of 'git@github.com:gibson042/canonicaljson-spec.git' into submodule path '/go/src/github.com/piedup/canonical-test/canonicaljson-go/canonicaljson-spec' failed
Failed to clone 'canonicaljson-spec' a second time, aborting

It's strange because cloning directly the sub repository (canonicaljson-spec) works.

It seems to be a permission issue, so unrelated to dep. Although if you have a clue it could help !

[piedup]

git clone https://github.com/gibson042/canonicaljson-spec works git clone git@github.com:gibson042/canonicaljson-spec.gitdoesn't work

The .gitmodules points to git@github.com:gibson042/canonicaljson-spec.git this is why the recursive call doesn't work. It doesn't work on docker container because there is no ssh key linked to GitHub.com hence it can't clone via ssh.

Thank you for your help.

[sdboyer]

you beat me to it 😄

[rodrigo-brito]

Solved! Same problem here. Dep has a problem with private repositories, if you use your SSH key for fetch, use the command: ssh-add -K ~/.ssh/id_rsa to add your key to the keychain.

[chalda]

@rodrigo-brito thanks, that fixed my issue on MBP! worked for me working against private enterprise repos

[ppxl]

Hi, is there a chance that this issue is resolved with a fix in dep?

This very scenario just happened to me, and I had no idea this was an ssh issue. Only a day later, a co-worker said that it could have something to do with ssh. And here I am.

Is dep really supposed to run endlessly instead of timing out with an error message?

Regards

[chalda]

hmm thats a good point @ppxl may be it does and its just set to some unreasonably high number that is longer than my patience? Or may be they are only catching errors from ssh. Either way thats something that should be checked for.

[yuchaoran2011]

I ran into the same issue. dep ensure works on my laptop but not in a Linux docker image. How can I make it work in Docker?

[dongsam]

check permission and owner of $HOME/.config, in my case solve using sudo rm -rf $HOME/.config when there are not important files