Get rid of GitHub Actions

n0toose commented 3 years ago

Guidelines

[X] I have read the guidelines.

Is your feature request related to a problem? Please describe.

As a project, we are not strictly bound to GitHub, but we also accept patches on SourceHut. However, we've been mostly forced to use GitHub and its UI, which is very often counterintuitive for our needs, because it allows us to test against Windows and macOS environments.

We decided that our primary platform of choice would be SourceHut, but we're still being limited to GitHub because of its multifaceted tendency to force you to stay on their platform.

Describe the solution you'd like

We should figure out which alternative to use. It should ideally be free.

Describe alternatives you've considered

SourceHut (builds.sr.ht) explicitly does not support proprietary platforms officially, but we can't just force everyone to move off them. That would be a deal breaker for the vast majority of our supporters, prospective users and current users. However, we should use it for obscure desktop platforms like FreeBSD and we should keep our Arch Linux CI.

Additional context

To make things worse, we've resorted to temporarily using GitHub Actions for our nightly builds. See https://github.com/tenacityteam/tenacity/issues/393.

This issue is not a duplicate

[X] I have confirmed this issue isn't a duplicate.

nbsp commented 3 years ago

I wouldn't be opposed to self hosting macOS and Windows builds. We have the budget for that.

n0toose commented 3 years ago

I wouldn't be opposed to self hosting macOS and Windows builds. We have the budget for that.

On a VPS or a machine of our own? I've seen huge security fails with self-hosted CI's and I'm worried about how we could possibly ensure that our builds will be clean and not tampered. Maybe we have to work on reproducibility a bit more.

Be-ing commented 3 years ago

Here are options for CI services for Gitea, from https://gitea.com/gitea/awesome-gitea#user-content-devops :

agola: Linux only
appveyor: no concurrent jobs for free. Paid plans are expensive for only 2 concurrent jobs.
AWS Cloud Integration(webhook-to-s3): gross, wouldn't be any better than using GitHub even if it met our technical needs
buildbot-gitea: works with Windows, don't think it works on macOS
buildkite-connector: agent works with Windows and macOS, however requires using their proprietary SaaS service to orchestrate self-hosted agents
Concourse: works with Windows and macOS. Unclear of the state of Gitea support.
drone: works with Windows, don't think it works on macOS
Jenkins: works on Windows and macOS.
Metroline: Linux only
mvoCI: Linux only
woodpecker: Not sure about Windows? No macOS it seems.

GitLab CI also works on Windows and macOS. So it seems our options are:

Jenkins
Concourse?
buildkite: semi-proprietary SaaS :-1:
GitLab CI: open core freemium :-1:

Mixxx used to use Jenkins and it was way clunkier to use than GitHub Actions. Though I see Jenkins now has Pipeline files that can be managed in the Git repository like any other CI service, so it may be worth considering more seriously.

Concourse seems like relatively immature software. I would be more comfortable going with something established like Jenkins, even if it is clunky.

nbsp commented 3 years ago

Codeberg is also working on CI, if I remember correctly.

Another point that's very important is the fact that currently, only builds.sr.ht works on email patches. This makes the mailing list effectively unusable for anything more than a documentation change, which is no better than removing it outright. This isn't sourcehut's issue, I don't think: we should look for a CI service that integrates well with git on email.

Be-ing commented 3 years ago

Codeberg is also working on CI, if I remember correctly.

Codeberg is just a Gitea host so everything in my comment above except for GitLab applies to Codeberg.

we should look for a CI service that integrates well with git on email.

I do not know if such a software exists. If it does, I doubt it works with Windows or macOS.

owenwastaken commented 3 years ago

what about gitlab?

Semisol commented 3 years ago

what about gitlab?

we already have GH and sourcehut, not a third one also see https://github.com/tenacityteam/tenacity/issues/477#issuecomment-898413404

Be-ing commented 3 years ago

I wouldn't be opposed to self hosting macOS and Windows builds. We have the budget for that.

Yeah, I think we could do self host CI with Jenkins. There are several conditions that need to be met though:

The machines need to be on a stable Internet connection.
Whoever hosts the machines needs to be available to troubleshoot them and/or give others remote access to do so.
The operating systems and toolchains of the runners need to be kept up to date.

nbsp commented 3 years ago

On Fri Aug 13, 2021 at 5:08 PM IDT, Be wrote:

Whoever hosts the machines needs to be available to troubleshoot them and/or give others remote access to do so.

From a security standpoint, none of us can really be trusted. I'll try contacting Fosshost and DigitalOcean to see if they can help us in any way.

Be-ing commented 3 years ago

Fossshost applications are currently suspended. DigitalOcean doesn't run Windows or macOS.

ghost commented 3 years ago

Since github is the only alternative that actually gives the builds you need, with the security we all need (this project has a target painted on it) and the transparency users need (this project has a bumpy past), why not keep using it? For example, I know of a handful of projects which are primarily hosted (and by this I mean, that's the only place where issues/patches/etc occur) elsewhere (eg gitlab) but also mirror to here and build here.

I feel like this thread is trying to solve a problem but it's not actually been stated what the problem is. I get why you mightn't want the project to 'live' here, but why not build here? (edit for clarity: this isn't an argument, it's a genuine question)

nbsp commented 3 years ago

I feel like this thread is trying to solve a problem but it's not actually been stated what the problem is. I get why you mightn't want the project to 'live' here, but why not build here?

The problem with GitHub Actions is that they're tied to GitHub. CI for contributions for Windows and macOS only works on PRs, rendering the mailing list basically unusable except for small documentation changes. By continuing to use GitHub CI, we are locked in.

ghost commented 3 years ago

Sorry I'm still not seeing it. Are you saying that the problem is that GH CI jobs wouldn't be automatically triggered if the commit was made elsewhere and mirrored here?

nbsp commented 3 years ago

The problem isn't with commits to master (or other branches), it's with people sending in patches and PRs. patches are only tested against builds.sr.ht, while PRs get both it and GitHub Actions tests.

That's because builds.sr.ht isn't tied directly to the rest of sourcehut: you can use it wherever you want. GitHub Actions, on the other hand, only works on GitHub.com.

ghost commented 3 years ago

Thanks for taking the time to explain it, now I get it :)

Would it be bad to be trapped on GH, for a very limited subset of purposes? That way, you could relax requirements on the other purposes.... Or are you keen to find a single solution for all purposes?

Be-ing commented 3 years ago

If we could get the build results of GitHub Actions automatically reported for branches elsewhere, that could work.

n0toose commented 3 years ago

If we could get the build results of GitHub Actions automatically reported for branches elsewhere, that could work.

We could have a bot push proposed patches to GitHub branches, but that could possibly leave room for abuse.

n0toose commented 3 years ago

Reason why I set this to high priority is because "how are we going to compile all images, sign them, and then officially release them?". This will come back later and bite us one way or another.

Deciding that something isn't "high priority" without being fully aware as to why, say I, consider it high priority can be shortsighted sometimes and it feels like you're overriding your own judgement while dismissing mine. Let's just talk to each other, please.

ghost commented 3 years ago

This will come back later and bite us one way or another.

Not to be rude but it's biting RIGHT NOW. As much as I appreciate your company, I'm not here for the laughs, people. I'm trying to find out how the **** I'm going to edit audio. I'm trying to figure out what to tell my friends to do about it when I tell them they can't use their old editor any more.....

I don't know either way. if getting rid of github actions is the way forward for that, but the whole consideration of the build process I'd agree is pretty high priority.

n0toose commented 3 years ago

Currently on a break, but we currently actually depend on GitHub Actions for builds, which you can find here: https://github.com/tenacityteam/tenacity/issues/393

@xcasxcursex

Be-ing commented 3 years ago

Deciding that something isn't "high priority"

Let's get rid of the "high priority" label. It is a useless cause of arguments. Everyone has different priorities. I think the only priority distinction that matters is "release blocker" or "not release blocker".

n0toose commented 3 years ago

Yeah, you're absolutely right, deleting the high priority and going to use in-progress as a stand-in. Would fit in this situation here as well, because that's something we're actively deliberating.

n0toose commented 3 years ago

Good idea to close this.

tenacityteam / tenacity-legacy