ci: move release process to GitHub actions

diegodelemos commented 4 years ago

Releasing through Travis is one of the main blocking factors when running Invenio sprints, all developers are submitting pull requests and our quota gets consumed, queuing release builds for long times (and consequently delaying other builds/releases which depend on it). If we move the releases to GitHub actions we would benefit from:

Faster releases, as the release builds will be in a different system (GitHub actions) than the one to run pull requests tests (Travis)
Gain experience and have an indicator regarding whether moving completely from Travis to GitHub actions would scale

ppanero commented 4 years ago

Some times we require releases by hand... having a script to release would be acceptable too? (maybe just for extreme situations?)

diegodelemos commented 4 years ago

This manual action would be just (correct me if I am wrong):

Juggling with Git: Merging to the correct branch (the PR, if there is any, should already point to the correct branch), tagging etc...
Using twine for the release: whose documentation is clear and concise.

In my opinion, this should be rather a documentation snippet than a script since it is just 2~3 commands long, plus, these commands can change (merging and tagging), since the nature of this is "extreme situations"/unusual cases.

ppanero commented 4 years ago

Agree on that :)

diegodelemos commented 4 years ago

Current stats

min/month: ~13,000 mins average queue time: 4.38 mins active repos: 59 builds/month: 746

Infor extracted from Inveniosoftware's Travis insights.

Comparison

Note: All data corresponds to the free/open source software plan for each system. Links to the sources are linked below.

System	min/month	concurrent jobs	self-hosted runners	can scale beyond releases?
GitHub actions	unlimited	20	yes	yes
Gitlab CI	2.000	Not found	yes	yes
Gitlab CI (CERN)	-	-	-	yes*
Circle CI	2.500 credits (~150 min/month)	1	no	no
Travis CI	unlimited	1	no	no

(*) Not sure since it would depend on requesting more resources internally at CERN if the limits are met.

Info sources:

GH actions pricing
GitLab CICD pricing, 📄 docs on how to do it, 🎥 short video tutorial
Circle CI pricing, credits info
Travis CI pricing

Considerations

If the system allows self-hosted runners we will always be able to scale even in the worst-case scenario, but if the chosen system can by default absorb the load:

It will be simpler, it will work out of the box
We will remove almost completely any doubts regarding security concerns since we would activate only the self-hosted runners when there is a usage peak.

CC @inveniosoftware/architects

ntarocco commented 4 years ago

Thank you for this investigation! 🚀 It looks like, from this table, that GitHub actions is a clear winner for our needs, with Travis just behind. Randomly checking some our builds in a few Invenio modules, each job takes from 10 to 20 minutes.

For CircleCI, this means: 2500 / 15 min = 166 builds Excluding Invenio sprints, if between ILS and RDM we create a couple of PRs per project per day day, and we run it a couple of times, it means: 2 projs x 2 PRs x 2 runs = 8 builds per day (IMHO, it is much higher in reality... but we can exclude weekends!)

We can basically run builds for 166 / 8 = 20 days...

ppanero commented 4 years ago

Plus the self-hosted runners... If needed, during sprints we can spawn a few machines (assuming puppetized setup for example).

inveniosoftware / invenio