search

ScottArbeit / Grace

Grace Version Control System
MIT License
545 stars 15 forks source link
cloud-native dapr developer-tools dotnet fsharp source-control version-control version-control-systems

readme

Grace Version Control System

grace (n) -

  1. elegance and beauty of movement, form, or expression
  2. a pleasing or charming quality
  3. goodwill or favor
  4. a sense of propriety and consideration for others [^grace]

Welcome to Grace.

Grace is a new, modern, cloud-native version control system.

Grace is easy to use, easy to understand, and consistently fast. And it's powerful, ready to handle large repositories and large file sizes.

Grace Server scales up by running on Kubernetes and massive PaaS services from large cloud providers.

Grace Client runs in the background, making it ambient, faster, and more valuable to your everyday work as a developer.

Grace connects you with others working in your repository, across the globe, in real-time, enabling new experiences and new ways of sharing.

to be clear:

Grace is new 🧑🏼‍🔬 and alpha-level right now. 🔥🧯 The parts that are implemented work, but there's much more to do. Grace is not ready to be relied on as a production version control system yet. It should not be used for anything other than its own design, development and testing, and for feedback.[^git]

Grace at NDC Oslo 2023

I gave my first conference talk about Grace at NDC Oslo 2023. You can watch it here:

FAQ

For a list of (mostly imagined) frequently asked questions, please see the Frequently Asked Questions page.

Design and Motivations

If you'd like to read about some of the design thinking and motivations behind Grace - topics like UX, performance, scalability, monorepos, Git, why F#, and more - please read Design and Motivations.

Vision

A lot of this remains to be built, but here's the vision for Grace v1.0:

No more fear

Stop being afraid of your version control system, and start enjoying it instead.

Grace is easy-to-use, easy-to-understand, and fast, with cool new features that let version control fade into the background while you're working, and help you remember where you were when you get interrupted.

There are fewer concepts to understand in Grace, so learning it is easy, and understanding what it's doing is simple. There's a simple-to-understand grammar, built-in aliases for common gestures, and common-sense defaults.

Grace is powerful source control, minus the fear.

Yes, it's fast

Great UX requires great speed. And Grace is fast.

Grace Server is designed to run on fast, cloud-based PaaS services for incredible scale and performance. Grace uses virtual actors as networked, in-memory data caches to maximize performance.

Grace CLI adds just milliseconds to the server response time for each command.

Grace's GUI apps will be platform-native, with all of the performance and stick-to-your-finger-ness that native apps have always had.

Grace Server can even precompute views and projections that you're likely to want, like diffs and directory contents, and garbage-collect them when they're no longer needed.

grace watch

Grace CLI includes a command - grace watch - that watches your working directory for changes, and keeps a live connection to Grace Server, connecting you in real-time to your team around the world.

grace watch takes action for you, whether that's automatically uploading and downloading new file versions, processing notifications from other repo users, or running custom actions that you create.

And don't worry about having yet-another application running in the background. grace watch is small and very quiet when nothing is going on.

Commits... and saves, checkpoints, and promotions

In Git, commit is an overloaded concept. It can mean:

And then you get the "squash" vs. "don't squash" debate. Sigh.

Grace simplifies this by breaking these usages out into their own gestures and events:

and, introducing:

Every save is uploaded, automatically

By default, grace watch uploads new file versions after every save-on-disk, along with a new snapshot of the entire directory structure of the repo, including new SHA-256 hashes.

It happens so quickly you don't even notice it.

And it gives you some very cool things, like:

Every change recorded

Grace is event-sourced. That means that everything that changes the state of the repository - every save and commit, every branch name change, every everything, is stored as a separate event.

As they're handled, they're sent to an event processor, which can log them in your choice of format and system. You can see the who / what / where / when of everything that happens in your repository, using your favorite event analytics and stream analytics tools, and even set up custom auditing and automation based on those events.[^stream]

Live, two-way client-server communication

When running grace watch, Grace uses SignalR to create a live, two-way communication channel between client and server.

This connection allows Grace to do All The Cool Things. Things like connecting you in real-time to everyone else working in your repository. Things like auto-rebasing. Things like watching for events in your repository, notifying you when you want to be notified, and running custom local actions if you want.

Imagine: there's a promotion to main, your branch gets auto-rebased on those latest changes, and then your local unit test suite gets run automatically so you immediately know if there's a problem.

Grace lets you share your code with team members effortlessly, around the world, for those times when you need another set of eyes on it, or just want to show them something cool.

Auto-rebasing keeps you up-to-date and ready to go, because...

Grace reduces merge conflicts

Merge conflicts suck. Finding out that you have one, when you thought you were already done with your work, is anxiety-inducing, and one of the most confusing parts of the user interface in Git. Grace helps you eliminate conflicts by keeping your branch up-to-date, automatically.

When your parent branch gets updated, within seconds, grace watch will auto-rebase your branch on those changes, so you're always coding against the latest version that you'll have to promote to.

Most-to-almost-all of the time, you don't even notice it. No errors, no conflicts.

When there is a problem, auto-rebase lets you find out right away.

You can fix it while you're in flow, and skip the conflict later.

AI to help resolve conflicts

And Grace doesn't just tell you there's a conflict. It uses AI to propose a resolution, and lets you accept it with a single click, or request that it tries again with a different proposal. You can even tell Grace to run a CI/CD pipeline on the resolution before proposing it to you.

C'mon, it's 2024. There's got to be some AI in here, right?

Let's shift left on promotion conflicts. Grace can't eliminate all of them, but it will reduce how often they happen, and it will proactively offer AI help to resolve them, so you don't have to worry about them ever again.

Personal branches, not forks

With Grace, there's no need for forking entire repositories just to make contributions. In open-source repos, you'll just create a personal branch against the repo.

You'll own your personal branch, and you can make it public or private. Your branch gets auto-rebased like any other branch in the repository, to keep your version up-to-date. When your change is ready you can submit PR's to get your personal branch's version promoted to a parent branch in the repo.

This is how I expect a large, open-source project in Grace to be: dozens of contributors, each with personal branches, working on a public project that remains securely controlled with ACL's. Everyone gets auto-rebased with every update to main. No networks of forks to manage, no entire copies of the repo. Just individuals working on the same repo, securely, together.

Simplified branching

Grace's branching strategy is called single-step and is designed to help reduce conflicts, and to make it easier to work on and promote code to shipping branches (like main).

Single-step branching is both easy-to-use and powerful enough to be all that you need to run your projects.

There's a separate page that describes it in more detail.

Grace is built for monorepos

At GitHub, I've been fortunate to work with customers who run some of the largest monorepos in the world. I've seen the challenges they face, and I've seen what they need to build and ship code to production.

Grace is ready for it.

Grace is designed to handle monorepos with ease. It's built to scale, and treats every repository, branch, directory version, and file version as a separate entity in a networked cluster of virtual actors. This design allows Grace to scale to handle arbitrarily large repositories, just as easily as it handles small- and medium-sized repositories.

So far, Grace has been tested on repositories as large as 100,000 files with 15,000 directories, with excellent performance, and no limits in sight.

Large, binary files welcome

Grace has no problem storing large files. Really large files. It's been tested with 10GB files - not that I think files that large belong in version control - and it should handle even larger files well.

Grace will let you specify how to handle those files, like only downloading them for the Design department, but not for Engineering. It's up to you.

Native GUI

Yeah, I said it.

Grace will have a native GUI app for Windows, Mac, Android, and iOS. (And probably Linux.)

Take Grace with you wherever you go. Merge conflict UI, Version History view, repository browsing, current status and more... all running at full native speed on your devices.

Web UI

Shipping a native app doesn't mean that we don't also need a great web UI.

Sometimes the best way to share information is using a URL.

CLI + Web UI + GUI... use Grace the way you want to.

ACL's, down to the file level

Grace goes way beyond just repository-level permissions. Grace uses OpenID and OAuth2 to integrate with your AuthN and AuthZ providers. Want to lock down specific paths in your repository to specific users and groups? With Grace, you'll be able to.

No stashing

Because grace watch automatically uploads new file versions after every save, and commands like grace switch ensure that everything in your working directory is preserved before switching to another branch or reference, there's no need to stash anything. Just switch and go.

Branch-level control of reference types

Plan your branching strategy in greater detail than ever before. Grace lets you decide which kinds of references are enabled in each branch.

For instance, main might enable promotions and tags, and disable commits, checkpoints, and saves.

Ordinary user branches might disable promotions, but enable everything else.

It's up to you.

Ephemeral checkpoints and saves

Checkpoints and saves are features to help you be more productive, but we don't need to keep them forever. Grace allows you to control how long they're kept are kept in each repository. 72 hours? One week? A month? Or maybe just "keep the last 1,000 saves". It's up to you.

Delete versions if you need to

Sometimes you need to delete a version, whether it's because a secret or password was accidentally checked in, or because of some other security issue. With the right permissions, you'll be able to remove bad versions of your code - with a permanent reference that says that you did.

Pre-rendered diffs

Every time you run grace promote, grace commit, or grace checkpoint, and every time grace watch uploads a new version of a file, Grace Server can optionally pre-render diffs for you to see, either in the CLI or in the Version History view. This makes seeing your ongoing changes in the Version History view instantaneous.

They're automatically aged out and deleted after a configurable length of time, so they don't just sit there wasting resources forever.

Optional file locking

If your repository includes large binary files, like edited video, graphics, or game artifacts, you need to make sure that only one person at a time edits each file. Grace will support optional file locking at the directory level to ensure that you don't have to re-do work.

(Don't worry... if you don't need file locking, you don't have to turn it on.)

A simple Web API

Grace Server is a modern Web API application. Like Grace's CLI, the Web API is easy to use, and easy to understand.

Grace ships with a .NET SDK, which is simply a projection of the Web API into .NET (and which Grace itself uses). SDK's for other platforms are welcomed as community contributions.

SHA-256 hashes

Grace uses SHA-256 hashes to verify that the files you uploaded, and the directory versions that went with them, are exactly the ones that get retrieved by clients. Grace will include a command to verify the SHA-256 hashes of all downloaded files and directory versions.

Local file cache

Grace repositories have a local cache of file versions that have been uploaded and downloaded. When running grace watch, Grace can download new file versions from multiple branches in the background so your grace switch commands run nearly instantly.

The local file cache is pruned regularly.

2048 characters

When you're running grace promote/commit/checkpoint/save/tag -m <some message>, the \<some message> part can be up to 2048 characters.

50 characters... I don't think so. Feel free to share details. The person you help might be future you.

Import from Git / Export to Git

Yes, we know... it's hard to let go. Grace will perform an initial import from a Git repo, and will export to a Git repo.[^gitexport]

Operations (if you're thinking about hosting your own server)

Grace Server is a modern, cloud-native Web API application. It will ship in a container on Docker Hub. (Of course.)

Grace Server is designed to be easy to deploy and operate. It runs on your choice of dozens of cloud-native databases, components and services, using Dapr, making it flexible and inherently scalable. Grace Server is stateless and scales up and down well using basic KEDA counters.

Project structure

Grace is written primarily in F#, and is organized into nine .NET projects.

An additional project, Grace.Load, is an experiment to create a load test for Grace.Server.

Project diagram

Solid lines indicate a .NET project reference. Dotted lines indicate network requests.

flowchart LR
    subgraph Server
    Grace.Server-->Grace.Actors
    Grace.Server-->CosmosJsonSerializer
    end

    subgraph Clients
    Grace.CLI-->Grace.SDK
    Grace.SDK-.->|HTTPS|Grace.Server
    Avalonia["(future) Grace.Avalonia"]-->|"(not yet started)"|Grace.SDK
    Blazor["(future) Grace.Blazor"]-->|"(not yet started)"|Grace.SDK
    end

    subgraph Utilities
    Grace.Load-->Grace.SDK
    end

    subgraph Tests
    Grace.Server.Tests-->Grace.Server
    end

    Grace.Actors-->Grace.Shared
    Grace.Server-->Grace.Shared
    Grace.CLI-->Grace.Shared
    Grace.SDK-->Grace.Shared
    Avalonia["(future) Grace.Avalonia"]-->|"(not yet started)"|Grace.Shared
    Blazor["(future) Grace.Blazor"]-->|"(not yet started)"|Grace.Shared

Code of Conduct

Our Code of Conduct is available here. The tl;dr is:

The name of this project is Grace.

Be graceful in your interactions.

Give grace to everyone participating with us.

Create something together that embodies grace in its design and form.

When in doubt, remember the name of the project.

Deployment

Grace Server will be shipped as a container, which will be made available on Docker Hub. Dapr's sidecar and actor placement processes are shipped as containers and are available on Docker Hub.

We intend to provide a Docker Compose template, as well as Kubernetes configuration for deployment, allowing for deployment to any major public cloud provider, as well as on-premises hardware.

[^grace]: Definition excerpted from https://www.thefreedictionary.com/grace.

[^git]: Grace currently uses Git for its source control, and runs Grace in the same directory as a means of testing. Officially self-hosting Grace's source code on Grace will, of course, happen when it's safe to.

[^stream]: One thing I'd like to do with the event log as a stream: detect invalid sequences and frequencies of events in Grace that would indicate bugs or attacks.

[^gitexport]: Grace will export the latest state of each branch into a git bundle file. Exporting an entire Grace repository, including history, to a Git repository probably won't be supported, but maybe someone else will write it. (There are, no doubt, a lot of edge cases to be found the hard way in that translation, and it's a very low priority item for me.) Live two-way synchronization between Grace and Git is a non-goal, for the same reason.