clone_repository from github.com fails with ed25519 key from v1.7.2

[howbazaar]

Opening #1143 as new issue as still fails with 1.11.1

Platform: linux x86_64 (ubuntu 20.04) Python: 3.9.12

Reproduction script: https://gist.github.com/howbazaar/b4e46806d0c5350623bf8d9ff49c6b4c

works with pygit2 v1.7.1 fails with v1.7.2, 1.9.1, 1.11.1

The script is a bit weird because it builds and runs the code the same way our production code does, and I was attempting to find a minimal reproduction of the problem.

Personally I had an older 4096 bit RSA SSH key, but newer users were using ed25519 type SSH keys as recommended by github's own docs.

I created some new keys:

ssh-keygen -t ecdsa
ssh-keygen -t ed25519

And added them to my github profile.

Was using poetry to manage versions inside a local virtual env to easily switch between versions of pygit2.

Failure looks like:

❯ poetry run python clone.py git@github.com:libgit2/pygit2 /home/tim/.ssh/id_ed25519
['clone.py', 'git@github.com:libgit2/pygit2', '/home/tim/.ssh/id_ed25519']
cloning into /tmp/tmpb_uq79bl
Traceback (most recent call last):
  File "/home/tim/stacklet/test/clone.py", line 58, in <module>
    main(sys.argv)
  File "/home/tim/stacklet/test/clone.py", line 47, in main
    gitrepo = pygit2.clone_repository(
  File "/home/tim/stacklet/test/.venv/lib/python3.9/site-packages/pygit2/__init__.py", line 241, in clone_repository
    payload.check_error(err)
  File "/home/tim/stacklet/test/.venv/lib/python3.9/site-packages/pygit2/callbacks.py", line 98, in check_error
    check_error(error_code)
  File "/home/tim/stacklet/test/.venv/lib/python3.9/site-packages/pygit2/errors.py", line 65, in check_error
    raise GitError(message)
_pygit2.GitError: Failed to retrieve list of SSH authentication methods: Failed getting response

ecdsa keys are fine

ldd for v1.11.1 of the pygit2 shared object file has

❯ ldd /home/tim/stacklet/test/.venv/lib/python3.9/site-packages/pygit2/_pygit2.cpython-39-x86_64-linux-gnu.so 
    linux-vdso.so.1 (0x00007ffe4bc53000)
    libgit2-44331e0d.so.1.5.0 => /home/tim/stacklet/test/.venv/lib/python3.9/site-packages/pygit2/../pygit2.libs/libgit2-44331e0d.so.1.5.0 (0x00007fb3678cb000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fb367890000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fb36769e000)
    libssl-6db5df62.so.1.1 => /home/tim/stacklet/test/.venv/lib/python3.9/site-packages/pygit2/../pygit2.libs/libssl-6db5df62.so.1.1 (0x00007fb367428000)
    libcrypto-f5871c65.so.1.1 => /home/tim/stacklet/test/.venv/lib/python3.9/site-packages/pygit2/../pygit2.libs/libcrypto-f5871c65.so.1.1 (0x00007fb366f71000)
    libpcre-8e472c69.so.3.13.3 => /home/tim/stacklet/test/.venv/lib/python3.9/site-packages/pygit2/../pygit2.libs/libpcre-8e472c69.so.3.13.3 (0x00007fb366cfb000)
    libssh2-f6863d91.so.1.0.1 => /home/tim/stacklet/test/.venv/lib/python3.9/site-packages/pygit2/../pygit2.libs/libssh2-f6863d91.so.1.0.1 (0x00007fb366aa9000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fb366a8d000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fb366a83000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fb367ea1000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fb366a7d000)

So it looks like it is still shipping with libssl and libssh2.

[jdavid]

Can you try with the wheels from https://github.com/libgit2/pygit2/actions/runs/4117199516 ? There we build and bundle the latest version of openssl (3.0.7)

[jdavid]

Here there are wheels for all supported Python versions https://github.com/libgit2/pygit2/actions/runs/4122340875

[tvb]

@jdavid As reported here I am seeing this same issue for pygit2 1.12.0 ed25519 keys seems to not be working. How can I help you to get this solved?

[jdavid]

@tvb You can build your own wheels, like this:

OPENSSL_VERSION=3.0.8 LIBSSH2_VERSION=1.10.0 LIBGIT2_VERSION=1.6.4 sh build.sh wheel bundle

Probably you will need to set AUDITWHEEL_PLAT as well. The generated wheel will be found in the wheelhouse/ directory.

Let me know if you find a version of OpenSSL that works for you, maybe try with the latest in the 1.1.x series

(Pull the latest version of pygit2, I've done a small fix to build.sh)

[tvb]

@jdavid In the meantime I got it working with ecdsa on 256 bit keys.

[howbazaar]

Just tested with 1.12.2, which i think was just released, but I have the same issue with "Failed getting response"

[howbazaar]

I think we are going to explore building our own binaries to get around this

[howbazaar]

Tested with 1.12.2 again and read the error message properly. Works fine on ubuntu 22.04. Had issues with lambda/3.11 base container, but a yum update -y in our Dockerfile fixed that issue.