Restrict the number of E2E jobs that can run in parallel

potential solution is using matrix strategy. See examples:

example solution from chatgpt:

To restructure this workflow so that all jobs starting with "e2e-" run in a matrix strategy with a max-parallel value of 5, follow these steps:

Consolidate the "e2e-" jobs into a matrix We will create a job matrix that dynamically runs the e2e- jobs while preserving the custom configuration for each job.
Define the matrix with max-parallel Use the max-parallel feature to control the maximum number of jobs running concurrently.

Update each e2e- job to work with matrix variables Here’s how you can restructure the workflow:


name: "Build Application"
on:
workflow_dispatch:
inputs:
  enable-unit-tests:
    description: "Unit Testing Enabled"
    type: boolean
    required: false
    default: true
  enable-e2e-tests:
    description: "E2E Testing Enabled"
    type: boolean
    required: false
    default: false
  enable-snyk-scan:
    description: "Snyk Scan Enabled"
    type: boolean
    required: false
    default: false
push:
branches:
  - main
  - 'release/*'

defaults: run: shell: bash

jobs: env-vars: name: Set Environment Variables uses: ./.github/workflows/zxc-env-vars.yaml with: custom-job-label: Set Environment Variables

code-style: name: Code Style uses: ./.github/workflows/zxc-code-style.yaml with: custom-job-label: Standard

unit-tests: name: Unit Tests uses: ./.github/workflows/zxc-unit-test.yaml if: ${{ github.event_name == 'push' || github.event.inputs.enable-unit-tests == 'true' }} needs:

code-style with: custom-job-label: Standard

e2e-tests: name: E2E Tests Matrix if: ${{ github.event_name == 'push' || github.event.inputs.enable-e2e-tests == 'true' }} needs:
env-vars
code-style strategy: matrix: e2e-test-type:
- { name: "Standard", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-standard-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-standard-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-standard-coverage-report }}" }
- { name: "Mirror Node", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-mirror-node-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-mirror-node-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-mirror-node-coverage-report }}" }
- { name: "Node PEM Stop", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-node-pem-stop-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-node-pem-stop-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-node-pem-stop-coverage-report }}" }
- { name: "Node PEM Kill", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-node-pem-kill-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-node-pem-kill-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-node-pem-kill-coverage-report }}" }
- { name: "Node Local Build", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-node-local-build-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-node-local-build-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-node-local-build-coverage-report }}" }
- { name: "Node Add", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-node-add-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-node-add-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-node-add-coverage-report }}" }
- { name: "Node Add - Separate commands", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-node-add-separate-commands-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-node-add-separate-commands-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-node-add-separate-commands-coverage-report }}" }
- { name: "Node Update", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-node-update-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-node-update-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-node-update-coverage-report }}" }
- { name: "Node Delete", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-node-delete-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-node-delete-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-node-delete-coverage-report }}" }
- { name: "Relay", npm-test-script: "test-${{ needs.env-vars.outputs.e2e-relay-test-subdir }}", coverage-subdirectory: "${{ needs.env-vars.outputs.e2e-relay-test-subdir }}", coverage-report-name: "${{ needs.env-vars.outputs.e2e-relay-coverage-report }}" } max-parallel: 5 runs-on: ubuntu-latest steps:
name: Run E2E Tests ${{ matrix.e2e-test-type.name }} uses: ./.github/workflows/zxc-e2e-test.yaml with: custom-job-label: ${{ matrix.e2e-test-type.name }} npm-test-script: ${{ matrix.e2e-test-type.npm-test-script }} coverage-subdirectory: ${{ matrix.e2e-test-type.coverage-subdirectory }} coverage-report-name: ${{ matrix.e2e-test-type.coverage-report-name }}

analyze: name: Analyze uses: ./.github/workflows/zxc-code-analysis.yaml needs:
env-vars
unit-tests

e2e-tests if: ${{ (github.event_name == 'push' || github.event.inputs.enable-unit-tests == 'true' || github.event.inputs.enable-e2e-tests == 'true') && !failure() && !cancelled() }} with: custom-job-label: Source Code enable-codecov-analysis: true enable-codacy-coverage: true enable-e2e-coverage-report: ${{ github.event_name == 'push' || github.event.inputs.enable-e2e-tests == 'true' }} secrets: snyk-token: ${{ secrets.SNYK_TOKEN }} codecov-token: ${{ secrets.CODECOV_TOKEN }} codacy-project-token: ${{ secrets.CODACY_PROJECT_TOKEN }}


Key Changes:
Matrix Creation: All the e2e- jobs are grouped into a matrix under the e2e-tests job.
max-parallel: 5: This ensures that no more than 5 jobs will run concurrently.
Matrix Variables: Each test case (e.g., "Standard", "Mirror Node", etc.) is represented as an object in the matrix with custom properties (npm-test-script, coverage-subdirectory, and coverage-report-name).
Job Execution: The reusable workflow (zxc-e2e-test.yaml) is invoked using the matrix.e2e-test-type variables.
This setup preserves the structure of the individual E2E tests while controlling their parallelism.

example solution pulling data from JSON

https://github.com/nbusseneau/cilium/blob/7178a98ef9f11bea0e32c68f0bc941d7063ee854/.github/workflows/conformance-ginkgo.yaml#L225-L236

more...

https://github.com/search?q=path%3A.github%2F**%2F*.yaml%20strategy%20matrix%20max-parallel&type=code

hashgraph / solo