Building a git remote helper

[chmac]

I have a working prototype now that takes a git repo and encrypts it by encrypting each object independently. I can push data to it and pull data back. That much works.

https://github.com/GenerousLabs/git-remote-encrypted

Now, I'm looking at how would it work if implemented as a git remote helper.

Firstly, reading this: https://git-scm.com/docs/gitremote-helpers

Then this is a very helpful guide: https://rovaughn.github.io/

Which in turn links to: https://github.com/git/git/blob/master/t/t5801/git-remote-testgit

The basics are:

• The helper gets invoked with 2 arguments
  • First: The remote name or it's URL if there is no name
  • Second: The remote URL
• git passes data to the stdin of the helper and reads from the stdout
  • Commands are batched and separated by a blank line \n\n
  • It sends a capabilities command first
  • We probably want to support fetch and push only at first
  • This actually means supporting several commands

That's about as much as I've understood thus far. Some additional useful reading:

• https://github.com/pandonetwork/pando
• https://github.com/noffle/git-remote-hypergit
• https://git.scuttlebot.io/%25ZdSn%2FxVxJ8B6464DFF%2BUw9gTjd0rtFZw3LHKqnFVgnE%3D.sha256/blob/e17c7e2d61266e282f1d6e262cf97e554db37bf0/index.js
• https://git-ssb.celehner.com/%25a4gRtAS8FLgWfxcM6SRFly2KnY9fdG8dWRAzgXsaxW0%3D.sha256/blob/95b35cbff5b43dfefeb32842b2f315b8e06f8ff7/index.js
• https://github.com/rovaughn/git-remote-grave
• https://github.com/cjb/GitTorrent/blob/master/git-remote-gittorrent
• https://github.com/anishathalye/git-remote-dropbox

[chmac]

Seems like after the capabilities command, git wants to get a series of refs from the remote. This is done via the list and the list for-push commands. The simplest response to this, which means that we don't know / have anything, looks like:

? refs/heads/master
@refs/heads/master HEAD

This means that refs/heads/master is a ref and that we don't know what sha1 it links to. Then the second line means that the HEAD "symbolic ref" points to the first line. Essentially this means "empty repo, no content".

[vHanda]

I was experimenting with this long ago and my idea was that the remote helper act like any other remote helper, except that when it's providing the 'objects', they are encrypted. So the remote will have all the correct references, and hashes the objects, except it wouldn't be able to verify the hashes as the git objects it would have would be encrypted.

I'll checkout your nodejs proof of concept and see what it is doing.

[chmac]

@vHanda My assumption is that if we encrypt the object contents, and the hashes don't match, then normal git hosts would reject the objects. Maybe you're thinking about using a custom backend to store the data?

[vHanda]

@chmac: Yes. This was for a custom backend.

[chmac]

@vHanda OK, that's where we're misaligned. I'm aiming to be able to use any standard git host as the backend.