gitfs: error message not descriptive enough when libgit2 was compiled without libssh2

[yrdevops]

We have Stash running ssh on a custom port. Gitfs_remotes doesn't seem to handle this very well. Running 2014.7.1, following doc for pygit2 I get:

Exception 'Unsupported URL protocol' caught while fetching gitfs remote ssh://git@gitserverurl.no:7999/yr/saltstates.git

 Exception 'Unsupported URL protocol' caught while fetching gitfs remote git@gitserverurl.no:7999:Yr/SaltStates.git

[rallytime]

Thanks for the report @yrdevops. I know there have been several fixes to gitfs at the HEAD of 2014.7, but I am not sure if this was one of them or not.

@terminalmage ping

[terminalmage]

This probably is not part of those fixes.

@yrdevops Can you run salt-run fileserver.update -l debug 2>&1 | grep gitfs_provider from the master and post the output? This will let me know which python frontend to git you are using.

[josephgimenez]

Hi @terminalmage

I'm running into this issue as well and may be able to provide the requested information.

Error: (This is while running salt with salt-master -l debug)

[DEBUG ] gitfs is fetching from ssh://git@prod-git.server.com/pm/server-scripts.git [ERROR ] Exception 'Unsupported URL protocol' caught while fetching gitfs remote ssh://git@prod-git.server.com/pm/server-scripts.git

fileserver.update grep: root@prcowsa001:~# salt-run fileserver.update -l debug 2>&1 | grep gitfs_provider [DEBUG ] pygit2 gitfs_provider enabled

salt version: root@prcowsa001:~# salt --version salt 2014.7.4 (Helium)

pygit2 version: root@prcowsa001:~# pip list | grep pygit2 pygit2 (0.22.0)

/etc/salt/master: gitfs_provider: pygit2 gitfs_remotes:

• ssh://git@prod-git.server.com/pm/server-scripts.git

/root/.ssh/config: Host prod-git.server.com Port 2222

So we're hitting this server via port 2222, which appears to match what @yrdevops was trying to accomplish.

It appears that may be a pygit2 limitation as the other plugins (dulwich, gitpython) work with regard to custom ports and the ssh:// protocol.

Please let me know if there's anything else I can provide to help.

[terminalmage]

OK, I know that @yrdevops said he was using pygit2 by my reasoning for having him/her run that command was to make sure that pygit2 was actually the provider that Salt was trying to use.

To my knowledge, pygit2 does not use one's ~/.ssh/config.

[dmurphy18]

@yrdevops can you retry pygit2 without relying on ~/.ssh/config

[josephgimenez]

Hi @dmurphy18, @terminalmage

I renamed my ~/.ssh/config file as ~/.ssh/config.bak then updated /etc/salt/master to hardcode the :2222 port within the repository reference as such:

/etc/salt/master config:

gitfs_provider: pygit2
gitfs_pubkey: /root/.ssh/id_rsa.pub
gitfs_privkey: /root/.ssh/id_rsa

gitfs_remotes:
  - ssh://git@prod-git.iad.server.com*:2222*/pm/server-scripts.git
  - ssh://git@prod-git.iad.server.com:2222/servers/prdemdb1.flidias.server.com.git
  - ssh://git@prod-git.iad.server.com:2222/servers/prshrdb1.flidias.server.com.git

when the salt master attempts to refresh its cache of the git repositories, it throws the following exception and stacktrace:

[ERROR ] Exception 'Unsupported URL protocol' caught while fetching gitfs remote ssh://git@prod-git.iad.server.com:2222/servers/prdemdb1.flidias.server.com.git Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/salt/fileserver/gitfs.py", line 1150, in update fetch = origin.fetch() File "/usr/local/lib/python2.7/dist-packages/pygit2/remote.py", line 221, in fetch check_error(err) File "/usr/local/lib/python2.7/dist-packages/pygit2/errors.py", line 56, in check_error raise GitError(message) GitError: Unsupported URL protocol

[terminalmage]

@asyncsrc I surrounded the config example in triple-backtick lines for readaibility, and noticed that you have asterisks in your first remote. Those probably shouldn't be there.

I made an SSH tunnel for localhost:2222 to github.com:22 and tested fetching from github using localhost:2222 (i.e. ssh://git@localhost:2222/terminalmage/salt.git) and was able to successfully fetch without any errors.

I tested both using 2014.7.1 and 2014.7.4.

[josephgimenez]

Hi @terminalmage I apologize for the confusion; I was attempting to embolden the port to emphasize i was now hardcoding the port within the config. those asterisks definitely are not within the config file.

Just out of curiosity, are you also using pygit2? If so, what version are you using? I noticed that there was a prerequisite for libssh as well that could be a factor, but I'm not sure how to tell if that's the issue or not.

I saw it's mentioned here: http://www.pygit2.org/install.html :: Libssh2, optional, used for SSH network operations.

I've installed what I presume are the prerequisite packages via apt-get then upgraded to the latest version of pygit2 via pip; however, I'm not sure that pygit2 has ssh support enabled, at least on my machine.

If this is in fact simply a configuration issue on my end, I do apologize for the potentially wasted effort.

[terminalmage]

@asyncsrc I tested with pygit2 0.22.0 and libgit2 0.22.0. libssh2 does need to be installed when libgit2 is built, so if you built libgit2 yourself (or it didn't dep on libssh2), then this might be the problem. The libgit2 on my laptop was built with libssh2.

[terminalmage]

If this is indeed the case, we should update our documentation, as well as providing a more descriptive error message.

[josephgimenez]

@terminalmage

so far I've found this post: https://github.com/libgit2/libgit2/issues/2665#issuecomment-71269299 that mentions that the libgit2 build will silently fail on libssh if the pkg-config package is not installed, which may have been what happened the first time I built libgit2.

I installed the pkg-config package (via: apt-get install pkg-config) then grabbed libgit2 version v0.22.2 from here: https://github.com/libgit2/libgit2/releases

ran the build commands here to get it built:

cd extracted_gitlib2_folder mkdir build && cd build cmake .. cmake -- build . --target install

I noticed it says during the cmake process.
-- checking for module 'libssh2' -- found libssh2, version 1.4.3

I then uninstalled pygit2 with "pip uninstall pygit2" then reinstalled with "pip install --upgrade pygit2"

now when running 'salt-master -l debug' i see:

[DEBUG ] gitfs received 0 objects for remote ssh://git@prod-git.iad.server.com:2222/servers/prdemdb1.flidias.server.com.git

that looks like a definite improvement :-) I would've liked it to say it received more than 0 objects, but it's certainly better than the 'unsupported protocol' error.. maybe.

Thanks for all of your help thus far. I really appreciate it.

[terminalmage]

The "received 0 objects" bit just means that the repo was up-to-date. Try stopping your master, clearing the gitfs cache (by removing /var/cache/salt/master/gitfs from the master), and starting up the master again. You should, on the initial fetch, see a non-zero number of objects fetched.

[josephgimenez]

Thank you @terminalmage

[DEBUG ] gitfs received 358 objects for remote ssh://git@prod-git.iad.server.com:2222/pm/server-scripts.git

We're in business. Thank you again for your help.

[terminalmage]

No problem, thank you for helping me get to the bottom of this. I will update our documentation and error messages to help point people to this solution.

[terminalmage]

@asyncsrc I rebuilt libgit2 with -DUSE_SSH=OFF and was able to duplicate the error. I've opened pull request #22800 to improve the error message.

Here's an example of the new error message:

[ERROR   ] Unable to fetch SSH-based gitfs remote ssh://git@localhost:2222/terminalmage/salt.git. libgit2 must be compiled with libssh2 to support SSH authentication.

Thanks again @asyncsrc!

[terminalmage]

I've renamed this issue to reflect the actual problem. I think we can close this out, now that I've updated the log error.