search

KSP-CKAN / NetKAN-Infra

NetKAN Infrastructure Repo
MIT License
3 stars 6 forks source link

[Feature] PyGit2 #148

Closed techman83 closed 4 years ago

techman83 commented 4 years ago

Problem

GitPython shells out to git, which has benefits and consequences. Benefit being that it closely matches what git can already do and you can run git commands if you need to. However you lose some visibility over the process and it's leaky, as noted in #142

Suggestions

PyGit2 has matured rapidly since the service was built and uses the libgit2 C bindings directly. We would have a lot more visibility over what's going on and wouldn't need to mess around with our environment as much (you can pass keys/user details etc straight into pygit2).

HebaruSan commented 4 years ago

The repo says https://www.pygit2.org/ is their documentation, but it's pretty horrible. Is there an actual intro/tutorial somewhere, with simple yet complete examples?

HebaruSan commented 4 years ago

(you can pass keys/user details etc straight into pygit2)

Unfortunately it looks like SSH connections still require creation of files:

https://www.pygit2.org/remotes.html#pygit2.Keypair

image

OOPS, missed this:

https://www.pygit2.org/remotes.html#pygit2.KeypairFromMemory

image

... except KeypairFromMemory doesn't seem to work. The fetch call just hangs, whereas making a Keypair from the file paths works fine. Oh, they have a 1.5-year-old report of that problem: libgit2/pygit2#800

techman83 commented 4 years ago

Yeah, I think stuff like that is why I stuck with GitPython. I didn't want to get bogged down yak shaving stuff like this. Now everything is kinda done, refactoring our git interactions is a little less daunting.

We might want to move our git interactions into a mixin class or something, with some tests. Then we can play around with pygit2 a little easier.

HebaruSan commented 4 years ago

In the meantime, should we look into periodically restarting the various containers? The latest pass is taking 9 hours to get from 'A' to 'O', as compared to the 10-20 minutes for the entire pass that we had a short while ago.

techman83 commented 4 years ago

2020-06-15_10-30-55

A refactor was done in #178, along with the work fixing #142 this is less of a priority. There is still more refactor work to do, but I think we can close this as an issue.