search

polydawn / repeatr

Repeatr: Reproducible, hermetic Computation. Provision containers from Content-Addressable snapshots; run using familiar containers (e.g. runc); store outputs in Content-Addressable form too! JSON API; connect your own pipelines! (Or, use github.com/polydawn/stellar for pipelines!)
https://repeatr.io
Apache License 2.0
68 stars 5 forks source link

Git2: Reducing git exec dependencies #107

Closed TripleDogDare closed 7 years ago

TripleDogDare commented 7 years ago

CI is complaining about the go-git libraries. Not sure what's up with that yet. This PR needs a little cleaning up yet. But it's far enough along that you can swap out the transport kind in repeat-thyself.sh and get correct outputs.

What does it do? Adds a git2 transmat which uses src-d/go-git library to do git operations. This is for testing purposes at the moment and is expected to be entirely compatible with and eventually replace the regular git transmat.

Goals 1) Not have dependencies on the host's git installation. This PR will get us most of the way there. The notable exception is that the default local repository transport will still shell out to git. 2) Be able to add features such as configuring a map of ssh keys to use for different repositories.

Performance There should be a small performance gain by fetching and moving less data around and not spinning up new processes. This is largely irrelevant though to our goals here and I didn't bother attempting benchmarks.

Try it Clear the repeatr cache (optional) sudo rm -rf /var/lib/repeatr/ Change git to git2 in repeat-thyself.sh

    "/task/repeatr/":
        type: "git2"

sudo ./repeat-thyself.sh Everything should run fine, with similar output logs and equivalent output hashes.

warpfork commented 7 years ago

Holy :tada:

This is gonna take me Several Time Units to review but i am So Excited