scylladb / scylla-manager

The Scylla Manager
https://manager.docs.scylladb.com/stable/
Other
48 stars 33 forks source link

Add an option to choose which tests to trigger in manager-build job #3856

Closed mikliapko closed 4 weeks ago

mikliapko commented 1 month ago

It makes sense to extend manager-build job with one more parameter where the required scope of tests to trigger after build can be selected.

It can be either:

mikliapko commented 1 month ago

I'd want to suggest the following configuration:

Three priority groups: p0, p1 and p2:

The group of tests chosen by default - p0.

In builds triggered manually it'll be the option to choose the group of tests.

@karol-kokoszka @Michal-Leszczynski @rayakurl please, let me know if you are okay with such test configuration.

p0:
  deb:
    main:
      - ubuntu22-sanity-test
    downstream:
      - ubuntu22-upgrade-test
p1:
  deb:
    main:
      - ubuntu22-sanity-test
    downstream:
      - ubuntu22-upgrade-test
      - sct-feature-test-backup
      - debian10-installation-test
      - debian11-installation-test
      - ubuntu20-installation-test
p2:
  deb:
    main:
      - ubuntu22-sanity-test
    downstream:
      - ubuntu22-upgrade-test
      - sct-feature-test-backup
      - sct-feature-test-backup-azure
      - sct-feature-test-backup-gce
      - debian10-installation-test
      - debian11-installation-test
      - ubuntu20-installation-test
      - debian10-upgrade-test
      - debian11-upgrade-test
      - ubuntu20-upgrade-test
      - ubuntu22-sanity-ipv6-test
      - older-enterprise-ami-sanity-test
rayakurl commented 1 month ago

@mikliapko - IMO please switch the first bullet with the second bullet: p1 - p0 + installation tests (debian10, debian11, ubuntu20, ubuntu22) and backup feature test; p0 - only ubuntu22 sanity + ubuntu22 upgrade test; p2 - p1 + backup feature test (GCE, Azure), upgrade test (debian10, debian11, ubuntu20), ipv6 test and sanity test on older Scylla Enterprise version.