git ps rr is slow when requesting large ranges

[ctsstc]

I'm not sure if it's just my instance, but when I'm requesting large ranges it seems that git ps rr is slow. It would be interesting if it could maintain a local branch and check if the patches within it were different, so if you add an additional patch to a range it only cherry picks that single patch out. Maybe there's a faster method of cherry picking out a range instead of one-by-one?

[drewdeponte]

Cherry picking ranges is generally fast with git. So it must be something additional we are doing in that process that is slowing things down. Not sure off hand what it is that would be making it slow. But request review of a series of patches is probably the most complicated command we have so it is definitely possible we have something in there that is not performant and is slowing things down.

We probably just need to go through and figure out what that thing is that is slowing things down.

Actually, I haven't been too happy with the performance even of git ps ls let alone any of the other commands. We should probably figure out how to profile a Swift command line tool to identify which functions are taking so long and then dig into improving those.

[drewdeponte]

There is also a fundimental performance constraint in the way this app is built. Basically it is bulit fundimentally by running the actual git command. In order to find the git command on the system it is also executing bash with a which git command. So every command it is paying that cost up front to launch any of the commands and then it is paying the cost of launching the git program as overhead any time it deals with a command that composes multiple git commands.

The ideal way of eliminating this overhead would be to use https://libgit2.org as a dependency. Then we would have this overhead.

I have even thought about reimplementing this in Rust using libgit2 as this is really just a spike to vet the idea and see if it really works and then we started using at as a daily driver.

[drewdeponte]

I did some research again last night to see if there are any decent bindings for libgit2 in Swift. Sadly https://github.com/SwiftGit2/SwiftGit2 looks like the leader and it doesn't even support SPM and has crappy docs.

So I think we are presented with the following options.

• continue w/ Swift
  • con - have to build / maintain own libgit2 wrapper to be able to eliminate shell & external command launch overhead
  • con - doesn't support easily support static linking & cross-compilation for distribution
  • con - doesn't have memory safety gaurentees
  • con - profiling tools for command line Swift apps aren't helpful
  • con/pro - we have things implemented in hackish spiked out fashion
• switch to Rust
  • pro - supports static linking & cross-compilation for easy distribution
  • pro - more performant than Swift in general
  • pro - has memory safety gaurantees
  • pro - has good libgit2 bindings https://docs.rs/git2/0.13.22/git2/
  • pro/con - to support building a UI later we would have to expose a library with a C interface so that an app could link to unless we built the GUI app in Rust with a cross-platform GUI toolkit
  • pro/con - we can't use the Swift implementation for individual commands while rewriting in Rust. Maybe we can but we have to implement a strategy to split out individual commands

Anyways, my instinct after the initial pro/cons breakdown is that long term strategy wise it probably makes sense to switch to Rust or at least start moving in that direction (breaking commands appart, etc.).

Would be interested to hear your thoughts @ctsstc .

[ctsstc]

Sounds like we should move it over to Rust! 😆 (Are you sure you don't want GoLang or gross raw bash scripts :P )

This would also drop the brew dependency for Swift which would be nice, but it sounds like we gain some other dependencies along the way, but they sound rather helpful.

Could we use a Unix socket to decouple a UI?

Moving this over sounds like a large process, but it sounds like the sooner the better. I would definitely try to hold off on additional features like profiling and try to implement those in Rust instead.

[drewdeponte]

Lol funny you say that. The initial version was written as individual bash scripts that I used to validate the workflow for myself. Then when I added file management and state tracking, and things like that to vet it with other people that is when I switch the spike to a Swift implementation.

In terms of other dependency we gain along the way I think they are good dependencies like libgit2 on the Rust front.

In terms of decoupling to support a UI. I wouldn't use a Unix socket in this case because it isn't a daemon process running. Instead I would simply have command line arguments that make the commands operate in a computer consumable fashion. For example maybe there is a cli switch to have git ps ls -j to dump the output in JSON so that a program could easily consume it.

I am not sure how large it actually is. I think a lot of the code that we currently have written will likely go away thanks to libgit2.

yeah, maybe I will play around with getting a rust version going this weekend and we can start to get a feel for the lift.

we can also do this with testing as we now have some validation and know how we want things to work for the most part

[ctsstc]

I haven't tried Rust out for myself, but this would likely be a great crash course to get acquainted.

I've also been listing ideas to give a brown bag on, and wrapping CLIs in a UI has been one of them.

[drewdeponte]

If you wanted to use this as an example I would be happy to help provide any guidance where I can as I have built UIs that wrap CLIs before.