Non explicit error: Unable to get repository

[emilevauge]

Hi!

I'm having an issue on traefik with aliasing a repository (that we forked). Here is the yaml configuration:

- package: github.com/vulcand/oxy
  vcs: git
  repo: git@github.com:containous/oxy.git
  version: 1ca0a134a818f7b8ea85de6e6554fe3312f144c2 

github.com/vulcand/oxy is the repository that I want to alias, using the fork github.com/containous/oxy.

The build error can be found here: https://travis-ci.org/containous/traefik/jobs/138409815#L371

Any idea?

[emilevauge]

I fixed it using repo: https://github.com/containous/oxy.git instead of repo: git@github.com:containous/oxy.git:

- package: github.com/vulcand/oxy
  vcs: git
  repo: https://github.com/containous/oxy.git
  version: 1ca0a134a818f7b8ea85de6e6554fe3312f144c2 

The issue is that it was working perfectly on my laptop, but not in a docker container. Indeed, glide was trying to clone the github.com/containous/oxy repo using SSH without any credentials.

1. The error given by glide was Unable to get repository. I think there is room for more details here :smile:.
2. I got this example in the documentation https://glide.readthedocs.io/en/latest/glide.yaml/:

- package: github.com/Masterminds/vcs
  version: ^1.2.0
  repo:    git@github.com:Masterminds/vcs
  vcs:     git

It would be better to use https instead of ssh in this example because the majority of us will use glide to build a project on jenkins/travis/circle, and will get stuck on the cryptic error message Unable to get repository.

WDYT?

[fanningert]

Hi there. I have the same problem with glide. Is there a solution (no workaround) for the problem?

[gavbaa]

Also experiencing this problem, and https isn't a reasonable option for us, we need to be able to check out the aliased imports over SSH as well, just like we do the rest of the projects that aren't aliased.

[franciscocpg]

Hi @gavbaa and @fanningert. The problem here is not with glide, ssh and aliases. Indeed the problem is that you don't have the appropriate ssh keys at the environment you are building.

[fanningert]

@franciscocpg Thanks for your answer, but why is it working sometimes and sometimes it doesn't work. (vendor directory is everytime cleared) and I thought for the git clone (only download) of a repository you don't need a ssh key.

[gavbaa]

I also disagree that this is an SSH key problem, because I'm doing it with completely public projects on Github.

I started a completely empty Go project, and with nothing but a directory structure and the following glide.yaml, get this issue.

glide.yaml:

package: git.jcasolutions.com/jca/traininglocker
import:
- package: github.com/jhoonb/archivex
  version: 60319bf17b608b56a3d34717d1501024c1a9f1eb
  repo: github.com/gavbaa/archivex
  vcs: git

Running glide up gives:

[INFO]  Downloading dependencies. Please wait...
[INFO]  --> Fetching updates for github.com/jhoonb/archivex.
[WARN]  Unable to checkout github.com/jhoonb/archivex
[ERROR] Update failed for github.com/jhoonb/archivex: Cannot detect VCS
[ERROR] Failed to do initial checkout of config: Cannot detect VCS

That seems like a legitimate alias problem to me.

[franciscocpg]

Hi @fanningert This is a log from a vagrant machine without an a ssh key trying to clone glide repository using ssh

git clone git@github.com:Masterminds/glide.git
Cloning into 'glide'...
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

So yes, you need it, :).

I take a look at the issue https://github.com/gogits/gogs/issues/3284 and https://gist.github.com/jakschu/f66b0b5863d06cf6d61577cd94bc4e41. Try to run with --debug flag, for example: glide --debug up. For me it looks more that you are getting some network problem when you are running glide up.

I just run glide up (using version 0.11.0 ) now at github.com/gogits/gogs without any problem.

glide up
[INFO]  Downloading dependencies. Please wait...
[INFO]  --> Fetching updates for github.com/go-macaron/captcha.
[INFO]  --> Fetching updates for github.com/go-macaron/gzip.
[INFO]  --> Fetching updates for github.com/go-macaron/csrf.
[INFO]  --> Fetching updates for github.com/go-macaron/session.
[INFO]  --> Fetching updates for github.com/Unknwon/cae.
[INFO]  --> Fetching updates for github.com/go-macaron/i18n.
[INFO]  --> Fetching updates for github.com/gogits/chardet.
[INFO]  --> Fetching updates for github.com/go-macaron/cache.
[INFO]  --> Fetching updates for github.com/Unknwon/com.
[INFO]  --> Fetching updates for github.com/gogits/cron.
[INFO]  --> Fetching updates for github.com/gogits/go-gogs-client.
[INFO]  --> Fetching updates for github.com/go-xorm/core.
[INFO]  --> Fetching updates for github.com/go-macaron/toolbox.
[INFO]  --> Fetching updates for github.com/codegangsta/cli.
[INFO]  --> Fetching updates for github.com/gogits/git-module.
[INFO]  --> Fetching updates for github.com/Unknwon/i18n.
[INFO]  --> Fetching updates for github.com/Unknwon/paginater.
[INFO]  --> Fetching updates for github.com/go-sql-driver/mysql.
[INFO]  --> Fetching updates for github.com/go-macaron/binding.
[INFO]  --> Fetching updates for github.com/go-xorm/xorm.
[INFO]  --> Fetching updates for github.com/issue9/identicon.
[INFO]  --> Fetching updates for github.com/kardianos/minwinsvc.
[INFO]  --> Fetching updates for github.com/lib/pq.
[INFO]  --> Fetching updates for github.com/mattn/go-sqlite3.
[INFO]  --> Fetching updates for github.com/mcuadros/go-version.
[INFO]  --> Fetching updates for github.com/microcosm-cc/bluemonday.
[INFO]  --> Fetching updates for github.com/msteinert/pam.
[INFO]  --> Fetching updates for github.com/nfnt/resize.
[INFO]  --> Fetching updates for github.com/russross/blackfriday.
[INFO]  --> Fetching updates for github.com/satori/go.uuid.
[INFO]  --> Fetching updates for github.com/sergi/go-diff.
[INFO]  --> Fetching updates for golang.org/x/crypto.
[INFO]  --> Fetching updates for golang.org/x/net.
[INFO]  --> Fetching updates for golang.org/x/text.
[INFO]  --> Fetching updates for gopkg.in/gomail.v2.
[INFO]  --> Fetching updates for gopkg.in/ini.v1.
[INFO]  --> Fetching updates for gopkg.in/ldap.v2.
[INFO]  --> Fetching updates for gopkg.in/macaron.v1.
[INFO]  Resolving imports
[INFO]  Fetching github.com/bradfitz/gomemcache/memcache into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching gopkg.in/redis.v2 into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/klauspost/compress/gzip into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/go-macaron/inject into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/go-xorm/tidb into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/ngaut/log into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/pingcap/tidb into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching gopkg.in/alexcesaro/quotedprintable.v3 into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching gopkg.in/asn1-ber.v1 into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/jaytaylor/html2text into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/shurcooL/sanitized_anchor_name into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching golang.org/x/sys/windows/svc into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching gopkg.in/bufio.v1 into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/klauspost/crc32 into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/juju/errors into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/klauspost/cpuid into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/golang/protobuf/proto into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/pingcap/tipb/go-tipb into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/petar/GoLLRB/llrb into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/pingcap/goleveldb/leveldb into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/rcrowley/go-metrics into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/twinj/uuid into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/golang/snappy into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/smartystreets/goconvey/convey into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/jtolds/gls into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/smartystreets/assertions into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Fetching github.com/gopherjs/gopherjs/js into /Users/fguima4/go/src/github.com/gogits/gogs/vendor
[INFO]  Downloading dependencies. Please wait...
[INFO]  Setting references for remaining imports
[INFO]  Project relies on 61 dependencies.

[franciscocpg]

Hi @gavbaa . Your repo config is wrong. Repo must be the repository address so the correct is

package: github.com/franciscocpg/test
import:
- package: github.com/jhoonb/archivex
  version: 60319bf17b608b56a3d34717d1501024c1a9f1eb
  repo: git@github.com:gavbaa/archivex.git
  vcs: git

I've made a test using version 0.11.0 and voila

[INFO]  Downloading dependencies. Please wait...
[INFO]  --> Fetching updates for github.com/jhoonb/archivex.
[INFO]  --> Setting version for github.com/jhoonb/archivex to 60319bf17b608b56a3d34717d1501024c1a9f1eb.
[INFO]  Resolving imports
[INFO]  Downloading dependencies. Please wait...
[INFO]  Setting references for remaining imports
[INFO]  Versions did not change. Skipping glide.lock update.
[INFO]  Project relies on 1 dependencies.

[gavbaa]

You are correct, that was the issue. Shouldn't a malformed repo entry for the specified VCS throw an explicit error to the user? Something that directly indicates the repo line is not valid?

[franciscocpg]

yes, I agree with you, but it's not a simple work. Glide works with a lot of VCS and would be necessary to map each of these address variations according to each protocol those VCS supports to do a proper validation. I think that @mattfarina can point out this for us. And PRs are always welcome too, :)

[emilevauge]

@franciscocpg that's what I was pointing in my previous comment https://github.com/Masterminds/glide/issues/476#issuecomment-226936940. Errors in glide should definitely be more explicit, because when you only get a Unable to get repository, you can't really guess what's wrong. I think it would be better to wrap errors using fmt.Errorf that would keep the original error. I don't get why there is a need to "hide" originals errors behind a generic one. WDYT?

[creshal]

Please give more detailed error messages. I keep getting the error on one machine, but SSH keys are set up and the configuration file isn't malformed, so there's something else broken and I'd like to know what.

[franciscocpg]

@creshal What comands are you trying to run?

[creshal]

glide up on a repository. Works fine on a bunch of Linux VMs, but fails on a windows VM with one specific repo. go get works fine with the exactly same repo URL, so I've verified that SSH etc. are working.

[franciscocpg]

Could you try glide --debug up and post the output here?

[creshal]

[ERROR] Update failed for internaldomain/somelib: Unable to get repository
[DEBUG] Output was: Klone nach 'C:\Users\creshal\.glide\cache\src\internaldomain-somelib.git' ...
Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.[DEBUG]       Unlocking internaldomain-somelib.git

Running git clone git@internaldomain/somelib.git manually in the same tab works.

The only difference I can think of is the SSH key – I'm not running ssh-agent on this machine, and I need to enter the key password on every SSH op. Is that not supported by glide for some reason?

[sakshamsharma]

I will give a weird report on this one. I too thought it has something to do with SSH keys, but this -->

• Builds fine on my laptop (without docker, on Gentoo)
• Fails inside docker on my laptop
• Works just fine on a Docker in Ubuntu 16.04
• Works fine when build on hub.docker.com

The errors are the same. Any thoughts?

Here are the debug logs for some of those which failed:

[WARN]  Unable to checkout github.com/klauspost/crc32
[ERROR] Update failed for github.com/klauspost/crc32: Unable to get repository
[DEBUG] Output was: Cloning into '/go/src/github.com/pclubiitk/puppy-love/.glide/cache/src/https-github.com-klauspost-crc32'...
fatal: unable to access 'https://github.com/klauspost/crc32/': Could not resolve host: github.com[DEBUG]    Unlocking https-github.com-klauspost-crc32
[WARN]  Unable to checkout github.com/miekg/dns
[ERROR] Update failed for github.com/miekg/dns: Unable to get repository
[DEBUG] Output was: Cloning into '/go/src/github.com/pclubiitk/puppy-love/.glide/cache/src/https-github.com-miekg-dns'...
fatal: unable to access 'https://github.com/miekg/dns/': Could not resolve host: github.com[DEBUG]  Unlocking https-github.com-miekg-dns

Oh, btw some github repos do work. Some just randomly fail.

Update: Weird. I looked here and realized that /etc/hosts file does not persist across RUN command. The debug logs showed that it may be a DNS issue (although it does build fine on another machine in the same internal network). I tried to fix by manually adding Github to the /etc/hosts file, and voila!

I managed to make it work. Here is the final command I ran:

RUN bash -c "echo 192.30.253.113 github.com >> /etc/hosts" && ./loop.sh

Even here, there were one or two failures. I fixed them by making loop.sh loop over glide install till it worked.

until glide --debug install;
do
    echo "Retrying"
done

[akhy]

completely agree with @emilevauge in my jenkins, running glide install inside docker build always failed due to lack of ssh keys

[bunyk]

Same issue when doing glide get github.com/__org__/__repo__ with private repository.

First it asks: [INFO] The latest release is v1.0.0. You are currently not using a release. Would you like Then [INFO] Would you like to track the latest minor or patch releases (major.minor.patch)? And then: [WARN] Download failed. [ERROR] Update failed for github.com/org/repo: Unable to update repository: exit status 1

[bunyk]

Anyone knows any workarounds?

[mattfarina]

If it's a private repo use the repo property in the glide.yaml file to set the repo for the package to be something like git@github.com/[org]/[repo].git and make sure git on that machine has access to the repo. This will work for private ones.