Make on-demand integration between EE and SDK

[NickVolynkin]

Make a workflow for integration testing of Tarantool CE with Tarantool EE and SDK.

Pull requests

CI:

• CE: https://github.com/tarantool/tarantool/pull/8038
• EE: https://github.com/tarantool/tarantool-ee/pull/311
• EE stub: https://github.com/tarantool/tarantool-ee/pull/317

Tools:

• https://github.com/tarantool/actions/pull/41
• https://github.com/tarantool/actions/pull/43

Experiments and testing:

• CE fork-based: https://github.com/tarantool/tarantool/pull/8091
• EE fork-based: https://github.com/tarantool/tarantool-ee/pull/316

Choice matrix

	fork-based PR	non-fork PR, label sdk-test	non-fork PR, no label
opened	run sdk-test-intro	no action *	run sdk-test-intro
synchronize	run sdk-test-intro	run sdk-test	run sdk-test-intro
reopened	run sdk-test-intro	run sdk-test	run sdk-test-intro
closed	no action	run sdk-test-cleanup	run sdk-test-cleanup
labeled with 'sdk-test'	no action	run sdk-test	-
unlabeled from 'sdk-test'	no action	run sdk-test-cleanup	-
workflow_dispatch	run sdk-test	run sdk-test	run sdk-test

^* when a PR is opened with a label, two events trigger: pull_request.opened, and immediately after it pull_request.labeled. Both will trigger a workflow run, but the first one will be canceled due to concurrency rules. This logic is vulnerable when there is more than one label on the PR. So the common rule is this: don't add other labels at the same time as 'sdk-test'.

Examples

Introductory comment

Introductory comment in a fork-based PR

Integration testing in progress

Commit message

integration-test: tarantool-master@nickvolynkin/sdk-test-workflow

bump submodule tarantool-master to commit

• https://github.com/tarantool/tarantool-ee/commit/c918c03a6bc98cbd972b0f7e79e3f90b5025e569, | \ which is a merge commit between | * https://github.com/NickVolynkin/tarantool-ee/commit/fcbbbdfce6b2a4b028ce6889cca008bf679697be (nickvolynkin/sdk-test-workflow) | and
• https://github.com/tarantool/tarantool-ee/commit/3f8cc97250a59bc6f878efd6c18b044696654b51 (master).

HEAD_REPO=NickVolynkin/tarantool-ee HEAD_REF=nickvolynkin/sdk-test-workflow HEAD_SHA=fcbbbdfce6b2a4b028ce6889cca008bf679697be BASE_REPO=tarantool/tarantool-ee BASE_REF=master BASE_SHA=3f8cc97250a59bc6f878efd6c18b044696654b51 MERGE_COMMIT_SHA=c918c03a6bc98cbd972b0f7e79e3f90b5025e569 PR_NUMBER=316

Reference

• https://docs.github.com/en/actions/learn-github-actions/contexts#github-context
• https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request
• https://github.com/tarantool/actions/tree/master/update-submodule
• https://github.com/marketplace/actions/repository-dispatch
• https://github.com/marketplace/actions/workflow_run-status
• https://github.com/actions/github-script
• Using inputs and secrets in a reusable workflow
• https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#masking-a-value-in-log
• https://cli.github.com/manual/gh_workflow_run

Subtasks:

• [x] Test-branch: create or update a test branch in tarantool/sdk.
• [x] Test-branch: remove the test branch
  • [x] when PR is closed/merged
  • [x] when label sdk-test is removed.
• [ ] Test-branch: think about branch name collisions, like when Alice and Bob push branches alice/foo and bob/foo.
• [x] Test-branch: fetch and use the merge commit (event.pull_request.merge_commit_sha)
• [ ] Test-branch: what if PR is not mergeable: use pull_request.head.sha, skip or raise an error?
• [ ] ~Test-branch: make a link back to the PR. Is it useful? It creates extra noise in the PR.~ Don't.
• [x] Submodule: select the target submodule by the PR target branch (github.base_ref value).
• [ ] Submodule: give a sensible output message when cannot find a proper submodule (unusual target branch, etc).
• [x] Feedback: create a comment with links and badges.
• [x] Feedback: update the comment on subsequent runs.
• [x] ~Feedback: when PR is opened from a fork, make a notice that sdk-test doesn't work with forks.~
• [x] ⭐ Feedback: post an informative comment when integration testing is available, but not enabled yet. (Conditions: ready for review (?), mergeable(?), non-bot(?)).
  • [x] with non-fork PRs
  • [x] with fork-based PRs
• [x] ~Feedback: add a notice about branch being not on top of stable branch (needs rebase). After merging such branch, integration tests may break even if they were okay on the branch.~ No need, as we're using pull_request.merge_commit_sha.
• [ ] Feedback: extract making job descriptions to a separate action, because we'll have to make a lot of them in the integration job in tarantool/tarantool.
• [ ] Feedback: final update upon merging the commit. Maybe give links to new branches/commits made from that commit?
• [x] Feedback: workflow badges get cached by the browser, so the results may be wrong. Resolved by adding &cachebust=${{ github.sha }}.
• [x] Feedback: comment has a link to the workflow's documentation;
  • [ ] link points to tarantool/tarantool@master.
• [ ] Workflow: only run on changes in source files (not docs or CI)
• [x] :star: Workflow: use workflow_dispatch to enable testing on forks
• [ ] Workflow: what if a developer wants to auto-update their branch to a specific branch from tarantool/tarantool? Maybe the rule is that we don't mess with the original branch, so they just update it to whatever they want.
• [ ] Workflow: handle changing the target branch of the PR? At least, check what happens.
• [ ] ⭐ Feedback: change commit status depending on the outcome of workflows in tarantool/sdk. This will require triggering these workflows with repository_dispatch and then getting results back. So, there are subtasks:
  • [ ] Turn off triggering workflows by pushing to specific branches
  • [ ] Trigger them with repository_dispatch
  • [ ] Change commit status depending on those jobs.

[NickVolynkin]

Resoved by https://github.com/tarantool/tarantool-ee/pull/311.

There's a lot more to do, and it will be done as a separate epic https://github.com/tarantool/tarantool-qa/issues/302