Move pipelines to more reproducible platform with less YAML

[TimothyStiles]

I really don't like the development workflow of CI/CD pipelines. They're hard to reproduce and usually a grab bag of random scripts custom written for each repo.

What I'd like is a programmatic platform where we can locally develop CI/CD pipelines without having to troubleshoot them directly against GitHub Actions. Something like Dagger comes to mind but I'm sure there are other tools we could use.

[carreter]

I'd be down to do this, would love to put CI/CD pipelines on my resume. Where would we host this?

[Koeng101]

What would they be used for? Why not just go test ./...?

[carreter]

CI/CD pipelines run before code is allowed to be merged, ensuring that we don't accidentally forget to test everything before it's committed to main. It also performs actions like uploading updated docs to the Go website, posting to social media, updating the test coverage badges in the README.md, along with anything else we want to configure it to do.

I'm very much on board with this, as it allows devs to more easily run all the testing/formatting/linting locally. It also allows us to avoid paying for CI/CD minutes on GitHub, and future-proofs poly in case we need to move it to another host.

[Koeng101]

CI/CD pipelines run before code is allowed to be merged, ensuring that we don't accidentally forget to test everything before it's committed to main

Well I know that, but I'm wondering what the git actions don't do right now that we want it to do.

uploading updated docs to the Go website

I think it updates itself

[carreter]

CI/CD pipelines run before code is allowed to be merged, ensuring that we don't accidentally forget to test everything before it's committed to main

Well I know that, but I'm wondering what the git actions don't do right now that we want it to do.

Nothing, except for perhaps the benchmarks you were suggesting in #366.

uploading updated docs to the Go website

I think it updates itself

Wait, what did you mean by this over on #346 then @TimothyStiles ?

get go docs to automatically pull in new tag on release. (very much wanted)

[carreter]

.take

testing the auto assign bot

[carreter]

.take

another test

[github-actions[bot]]

Thanks for taking this issue! Let us know if you have any questions!

[github-actions[bot]]

This issue has had no activity in the past 2 months. Marking as stale.

[carreter]

@ethanholz I know you had said something on the Discord about helping out with CI/CD stuff. Need to get this done before #423 , do you have any suggestions?

[ethanholz]

So I think there are a couple ways to come around to this. A tool I like using is https://github.com/nektos/act. It uses docker under the hood to test your GH actions locally and makes it easy for non-Git specific actions.

For simplifying the test->CI->release pipeline, using a tool like just (or make) can really help to mirror commands run in CI onto local machines. Furthermore, using a tool like make allows you to update what is run under the hood without changing how devs/users might run it locally.

I think that sticking with GitHub actions would work really well here but writing a spec for the local pipeline would help to make them easier to work with.

[carreter]

@TimothyStiles what are your thoughts on making our test -> CI -> release pipeline be based on make or just? I don't have a particular preference between the two - make will be much more portable, but just syntax seems a lot nicer.

I'll also include a writeup on how to use act in the repo readme.

[carreter]

After some discussion with @ethanholz on the Discord, I'm leaning toward just - even though it's not as standard as make, it seems a lot easier to work with and make does not come pre-installed on most systems as I had thought it was.

[carreter]

Fixed by #429.