Arlodotexe / strix-music

Combine any music sources into a single library. It's your music. Play it your way.
http://www.strixmusic.com
139 stars 4 forks source link

Distributed build validation via Git + IPFS + PowerShell #201

Closed Arlodotexe closed 2 years ago

Arlodotexe commented 2 years ago

This has been moved to https://github.com/Arlodotexe/brain-dump/issues/1

Background

Strix has pledged to make an effort to keep the project "perpetually preserved". Put another way, we should avoid relying on things with a single point of failure. The repo should contain everything needed to operate the project, and a release should contain a repo backup + all dependencies expected to be available to build the project, published to IPFS.

In order to keep things resilient, we use IPFS extensively to back up and host the content on our website (docs, WASM app, app binaries, build dependencies, source backup, published nuget packages, etc)

As a result of all this, Strix has a lot of build scripts. They contain everything needed to create a release, and there's a guide[1], [2] to help string them together into a working solution that can download build dependencies, generate documentation, generate our website, create release tags, generate changelogs, and more (with a few more coming in #203 and #202).

Problem

Build validation is done 100% in Azure DevOps.

For something that should be the equivalent of "yes this commit built successfully", DevOps and GitHub Actions may be extremely convenient, but it's complex, expensive and doesn't play into the spirit of Git.

Further, git itself has no built-in features that enable contributors to verify "yes, this commit built successfully".

We can do better.

Proposed solution

(proposal has been amended, see comments) (Prerequisite: #203)

Leveraging our existing build scripts and Git's already decentralized nature, we can combine Git with IPFS to create a distributed build validation system.

Since PowerShell is used in a lot of CI scenarios (and we've already leaned into it because of this), we'll be developing a PowerShell Module designed to work with Kubo.

First draft: Whiteboard (3)

Scripts

Node settings

Other

Arlodotexe commented 2 years ago

Proposal amendments

After more research into the inner workings of Git, I've found that we may be able to lean on Git even more, and massively simplify our setup in the process.

Permission management

It's a solved problem

Most services like GitHub and Azure DevOps already come with built-in permission management for the upstream repository, for both Branches and Tags.

Preserving validated commits

Tags, branches, and you

Git tags are only done against a specific commit, and cannot point to an branches. This means it's on us to make sure a built/validated commit is preserved in git history when merging branch.

A commit's hash is an immutable object that represents a set of changes, plus additional git information such as author, time, messages, etc, and a link to the parent commit, meaning it indirectly references all previous changes as well. It accurately represents the entire current state of the repo.

Can we build it?

I've verified the following behavior:

Yes, we can

No matter what merge strategy is used, we have the ability to preserve arbitrary information in tags, as long as the code exists on the remote.

The only caveat is that unless we know what merge strategy is going to be used, we'll have to create tags for both the upstream merge commit and a squash commit. We can make this part of the config on the "build validator" side of things.

I'll have an updated graphic soom:tm:., once we're done iterating.

Long term considerations

IPLD Integration

What if we went all in?

If we had unlimited time and resources, what would we make?

This has been moved to https://github.com/Arlodotexe/brain-dump/issues/2

Arlodotexe commented 2 years ago

Since the proposal has obviously gotten out of hand, I've created a new brain dump repo and moved them over.

In order to keep moving Strix forward, I don't plan on working on these things until much later, unless it becomes a requirement.