Create a github workflow for PR testing

[smlambert]

As we have recently encountered network issues (https://github.com/eclipse/openj9/issues/9560), we are very short of xlinux/mac machines and do not have capacity to adequately cover PR testing, we can put a workflow in place to help cover the gap.

2 github actions we will use for this workflow file (recently demonstrated in the OpenJ9 community call):

• build-jdk-openj9 - used to build openj9 (and can use the contributors personal repos for openj9, openj9-omr, and the jdk-jdk{Version} extensions repo).
  Note: this action currently resides in @sophia-guo personal repo, but we will eventually plan to move it to an official repo site, this move does not need to block the creation and use of this new workflow.

• run-aqa - used to run any of the test groups that are run at the openj9 and/or adoptopenjdk projects.
  Note: we can focus on sanity.functional to start with, and ask @pshipton and other committers for input re: what test targets to launch against a particular PR. In terms of trigger, we can start with a push to master as the trigger, but can also discuss other ways one might trigger this, once we have an initial workflow in place.

related: https://github.com/eclipse/openj9/issues/9547

[pshipton]

we can start with a push to master as the trigger, but can also discuss other ways one might trigger this

I'll note that master is supposed to always be in a working state, with changes pre-tested before they are pushed there, so we'll want to have that discussion.

Also, not to knock this, but fyi someone noticed there is a 2000 min (33.3 h) time limit per month in the free tier, which we will quickly reach considering each build and test is ~2+ hours linear time.

[smlambert]

Yep, we should also look at what enterprise gets you $21/month for 50,000min per month, but fair enough, we can just fix our infra problems and not go this route.

[smlambert]

At free tier, we can do 16 PRs per month, at enterprise tier we can do over 400 PRs per month.

[DanHeidinga]

How does github determine the 2000 minutes? Is that shared across the Eclipse foundation? Or per repo?

I wasn't clear on that from my quick glance thru the documentation

[smlambert]

Or per developer... wink! I believe its per public repo. See https://help.github.com/en/github/setting-up-and-managing-billing-and-payments-on-github/about-billing-for-github-actions

Also note that the administrator of the repo can set rules around what types of actions can be run...

When I was at DevNexus, I spoke with someone from another Eclipse project that was using workflows heavily, I am guessing they have something more than free tier, but I did not think to ask.

[smlambert]

There is also a minute multiplier for macOS/win apparently. linux 1 min == 1 min, macOS 1 min == 10 min windows 1 min == 2 min

[pshipton]

The page mentions multiple times "GitHub Actions usage is free for public repositories." so maybe we don't have a limit. I read it as the 2000 min limit is for private repos.

[pshipton]

we can start with a push to master as the trigger, but can also discuss other ways one might trigger this

IMO it should be triggered via a comment, like the existing PR testing, if that is possible. So a committer can trigger it on demand once initial reviews are completed and there are no dependencies.

There is a issue_comment event with activity created, but like current PR testing we'd have to parse the comment and look for keywords before triggering the action.

Otherwise, they aren't quite right but

• if there is a way to trigger on review approval, but I didn't see that
• there is a pull_request event with a ready_for_review or review_requested activity that may suffice, although it could run too many builds, and builds that can't possibly work without changes from other repos.

[smlambert]

The exercise is to offload some testing onto Github runners, perhaps we should not target the PR testing, but rather some of the nightly testing, while we discuss the various ways we could or should trigger PR testing.

Nightly can be triggered by a schedule, example:

on:
schedule:
  - cron:  '*/15 * * * 1-5'