Revisit support for Git

[bummoblizard]

As seen from #10, #24, #56, Git is a well requested feature. But one major problem is Git's GPL license, making it incompatible with App Store.

I wonder if there is any attempt to compile Git to WASM, enabling the ability to side load Git from user's side. This could bypass the regulatory issue.

[holzschu]

WASM does not have support for sockets or networking, for now. This will make porting git to WASM difficult. Although I could add socket support to WASM, the way I did add local file support.

In the next version (currently being uploaded) I am including a version of git based on Dulwich (a pure python package): https://www.dulwich.io It's not as complete as git, but I think it will be better than nothing.

The best candidate for a port of git is libgit2: https://libgit2.org , combined with objective-C bindings: https://github.com/libgit2/objective-git

[holzschu]

I gave it another try. git seems like it could compile to wasm, except it needs gethostbyname(), which is not part of the WASI SDK (for good reasons: there's no way to emulate that using JS).

[personalizedrefrigerator]

I've gotten git commit, git add, and git rm working using Dulwich.

I'm using this wrapper around Dulwich's CLI:

```py #!python import dulwich.cli import dulwich.repo from dulwich import porcelain import sys import argparse from argparse import ArgumentParser # Map (commandName) -> GitOverrideCommand OVERRIDE_COMMANDS = {} THIS_PROG = sys.argv[0] LONG_DESCRIPTION = \ """This implementation of `git' is intended for use in \ a-Shell. It currently uses Dulwich (see https://www.dulwich.io/) \ as a backend; however, while Dulwich provides its own command \ line interface, not all commands work well with `a-Shell'.\ For help with a specific command: git command_name --help """ NOTICE_OVERRIDDEN_COMMAND = \ """This command is overridden by a-Shell's git CLI.""" class GitOverrideCommand: argParser = None shortHelp = "No short help" help = None def printHelp(self): if self.argParser != None: self.argParser.print_help() elif self.help != None: print(self.help) else: print("No help available") return 1 def run(self, args): raise Exception("Not implemented") class ListCommandsCmd(GitOverrideCommand): shortHelp = "Lists available commands" def run(self, args): if len(args) > 0: return self.printHelp() print("Overridden commands:") for cmd in OVERRIDE_COMMANDS: print(" %s: %s" % (cmd, OVERRIDE_COMMANDS[cmd].shortHelp)) print("Commands from Dulwich CLI:") # -a flag: Print available commands. dulwich.cli.cmd_help().run(["-a"]) return 0 class AddCommand(GitOverrideCommand): shortHelp = "Stage changes" def __init__(self): self.argParser = ArgumentParser( prog = THIS_PROG + " add", description="Track changes", epilog=NOTICE_OVERRIDDEN_COMMAND ) # self.argParser.add_argument('-u', # help="Unstage files that have been deleted." # ) self.argParser.add_argument("files", nargs=argparse.REMAINDER) def run(self, args): args = self.argParser.parse_args(args) porcelain.add(paths=args.files) class RemoveCommand(GitOverrideCommand): shortHelp="Untrack files" help="Remove the given files from the index." def run(self, args): porcelain.remove(paths=args) class CommitCommand(GitOverrideCommand): shortHelp = "Record staged changes" def __init__(self): self.argParser = ArgumentParser( prog = THIS_PROG + " commit", description="Record a snapshot of all tracked files in the repository.", epilog=NOTICE_OVERRIDDEN_COMMAND + """ This command does not support hooks! """.rstrip() ) self.argParser.add_argument("-m", "--message", required=True, help="Label the commit with the given message." ) # self.argParser.add_argument("-a", "--all", action="store_true", # help="Stage all unstaged changes." # ) # self.argParser.add_argument("--amend", action="store_true", # help="Edit the most recent commit." # ) def run(self, args): args = self.argParser.parse_args(args) repo = dulwich.repo.Repo(".") repo.hooks = {} repo.do_commit(message=args.message.encode('utf-8')) return 0 ## Parse arguments intended for the root command (i.e. git). ## Returns a map with the keys command and command_args. The first ## is a string, the second is a possibly-empty array of arguments ## passed to the command. def parseCmdlineArgs(argv): args = ArgumentParser( description="A simple commandline interface for Git.", epilog=LONG_DESCRIPTION ) args.add_argument("command", help="The command to run. For a list of valid commands, try the command list-commands.") args.add_argument("command_args", nargs=argparse.REMAINDER) result = args.parse_args(argv) if not result.command: args.print_help() sys.exit(1) return result ## Commands we handle, rather than passing to Dulwich's default ## CLI. OVERRIDE_COMMANDS = { "list-commands": ListCommandsCmd(), "commit": CommitCommand(), "add": AddCommand(), "rm": RemoveCommand() } if __name__ == "__main__": args = parseCmdlineArgs(sys.argv[1:]) command = args.command if args.command in OVERRIDE_COMMANDS: sys.exit(OVERRIDE_COMMANDS[command].run(args.command_args)) else: try: dulwich.cli.main(sys.argv[1:]) except Exception as ex: sys.stderr.write("Exception thrown by Dulwich backend!\n") sys.stderr.write(str(ex) + "\n") sys.exit(1) ```

After I get it to work better (e.g. fix git reset and add the -u argument to git add, have git commit support --amend, open Vim when no message is specified, etc.), I intend to submit a pull request.

[holzschu]

Thank you very much for this hard work! Please see also issue #170: if we include git as a command, other commands, like vim, will call it, and it may break things down. It seems the difficult git plugin was vim-gitgutter, which you can use a test suite.

[personalizedrefrigerator]

It looks like a WASM version of git already exists (uses libgit2 compiled to WASM): https://github.com/petersalomonsen/wasm-git

[holzschu]

Thanks for the link. It is based on Emscripten WebAssembly, and so far a-Shell only works with WASI WebAssembly (which does not mean it won't work, but it is not tested yet). Supporting Emscripten is a whole new project (it assumes node is present, and I'm not certain about how it loads extra files).

It is easy to compile libgit2 to native iOS (or probably WebAssembly, but native iOS is easier), the problem is with the clients. There are very few git clients (that I know of) that are both based on libgit2 and have all the command line options of the standard git command line tool.

[personalizedrefrigerator]

It looks like wasm-git does expose (at least some) of the git command-line options:

https://github.com/petersalomonsen/wasm-git/blob/5119c56b5ca51423548c2f0729b180b2681590ef/examples/example_webworker.js#L16

I don't see where this is defined in their source repository though... Edit: See next comment. The client is defined in libgit2's example directory.

[personalizedrefrigerator]

I think it's a part of libgit2: https://github.com/libgit2/libgit2/blob/main/examples/general.c See below. main(argc, argv) is in lg2.c, not general.c.

I think libgit2/examples/* implement a simple git client... and this is what I think wasm-git is using.

Edit 1: This could be better than a Dulwich-based client -- Dulwich seems to start a background daemon that sometimes gives "Too many python interpreters running" errors (though this may have also been due to gitgutter's usage of the git command). Additionally, I've been having trouble getting some of Dulwich's commands (e.g. stash push/pop) to work.

Edit 2: main is in lg2.c!

[holzschu]

I think it would be useful to have both commands in a-Shell (your script above dulwich and lg2). Neither is going to work as a drop-in replacement for git, but each of them can be good enough for some uses. You can even call lg2 from Python for some difficult cases.

[NightMachinery]

Do any of these work with LFS files?