Implement "super" workflow to define inter-workflow dependencies

[stephanosio]

Implement a "super" workflow that includes individual CI workflows in order to facilitate defining the dependencies among the workflows.

This is necessary so that we can ensure the twister workflow (i.e. the most resource intensive one) does not start until all the basic compliance checks pass.

Note that this will require some major refactoring of the current CI workflows.

See https://docs.github.com/en/actions/using-workflows/reusing-workflows

[stephanosio]

Re: priority change, see https://github.com/zephyrproject-rtos/infrastructure-private/issues/22#issuecomment-1115096105.

[stephanosio]

Note that, with zephyr-runner v2, the ccache data will be stored in the local node cache as well as in the central Redis cache (i.e. AWS S3 secrets are not required); so, the twister workflow will no longer need to run on pull_request_target.

With this, it should be possible to chain compliance check and code build workflows in single pull_request event.

[stephanosio]

Brainstorming workflow re-organisation strategies

• Classify the existing workflows into two categories:
  • "Helper" Workflows
    • These are independent workflows that perform "helper" tasks such as assigning PR/issue categories and cleaning up stale issues.
    • They can be triggered standalone (i.e. not as part of another "super" workflow) and do not have any inter-workflow dependencies.
  • "Validation" Workflows
    • These workflows perform "validation" tasks such as coding guidelines check and running tests.
    • Each validation workflow is a reusable (workflow_call) workflow that can be invoked inside "super" validation workflows.
• Prefix the workflow file names with the above category names (e.g. helper-*.yml, validation-*.yml).
• There are three different types of "super" validation workflows:
  • Pull Request Validation Workflow (validation-main-pr.yml)
  • Push Validation Workflow (validation-main-push.yml)
  • Full (on schedule) Validation Workflow (validation-main-full.yml)

---

• Helper Workflows

  • Assigner (helper-assigner.yml)
  • Backport (helper-backport.yml)
  • Bug Snapshot (helper-bug-snapshot.yml)
  • Merge Blocker (helper-merge-blocker.yml)
  • Greeter (helper-greeter.yml)
  • Manifest (helper-manifest.yml)

• Pull Request Validation Workflow (validation-main-pr.yml)

  • Phase 1 (Compliance Checks)

    • Compliance Check (validation-pr-compliance-check.yml)
    • Coding Guidelines Check (validation-coding-guidelines-check.yml)
    • Backport Issue Check (validation-backport-issue-check.yml)
    • License Check (validation-license-check.yml)

  • Phase 2 (Tooling Validation)

    • General Script Test (validation-general-script-test.yml)
    • Pylib Test (validation-pylib-test.yml)
    • Devicetree Script Test (validation-devicetree-script-test.yml)
    • West Command Test (validation-west-command-test.yml)
    • Twister Test (validation-twister.yml)
    • libc Error Number Check (validation-libc-errno-check.yml)

  • Phase 3 (Zephyr Validation)

    • BabbleSim Test (validation-bsim-test.yml)
    • Documentation Build (validation-doc-build.yml)
    • Zephyr Test with GCC (validation-zephyr-test-gcc.yml)
    • Zephyr Test with Clang (validation-zephyr-test-clang.yml)

• Push Validation Workflow (validation-main-push.yml)

  • Phase 1 (Tooling Validation)
  • Phase 2 (Zephyr Validation)

• Full Validation Workflow (validation-main-full.yml)

  • Phase 1 (Tooling Validation)
  • Phase 2 (Zephyr Validation)