canonical / checkbox

Checkbox
https://checkbox.readthedocs.io
GNU General Public License v3.0
30 stars 45 forks source link

Some status of cases are still be non-blocker even apply blocker to .* to test plan #1492

Open baconYao opened 2 hours ago

baconYao commented 2 hours ago

Bug Description

In PR#1206 Certification team made every test job in client-cert-iot-.* test plans as cert blocker according to a decision made a few years ago which we have been using to judge the bug importance since then (All checkbox jobs being executed for the IoT project is a cert blocker)

Therefore, QA team started implementing this rule in each project. However, we observed that the status is still be non-blocker of some cases who are from base provider.

Here is the simple PR to implement this feature. What I did is like below (nest the cert test plan and have certification_status_overrides field be defined):

bootstrap_include:
    baoshan_device_soc
include:
nested_part:
    .
    .
    com.canonical.certification::client-cert-desktop-22-04-automated
    .
    .
certification_status_overrides:
    apply blocker to .*
exclude:
    .
    .
    .

To Reproduce

  1. Have a test plan that nests the com.canonical.certification::client-cert-desktop-22-04-automated
  2. Add certification_status_overrides field
  3. Sideload this test plan
  4. Checkbox remote runs this auto plan and choose the miscellaneous and info sets then executing it.

Environment

Relevant log output

No response

Additional context

No response

syncronize-issues-to-jira[bot] commented 2 hours ago

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/CHECKBOX-1577.

This message was autogenerated