canonical / checkbox

Checkbox is a testing framework used to validate device compatibility with Ubuntu Linux. It’s the testing tool developed for the purposes of the Ubuntu Certification program.
https://checkbox.readthedocs.io
GNU General Public License v3.0
33 stars 50 forks source link

Fix snap refresh verification job dependency (bugfix) #1519

Closed pieqq closed 1 month ago

pieqq commented 1 month ago

Description

When implementing changes for the gadget/snapd/kernel snap refresh, one of the job dependencies was not updated properly.

The snapd/reboot-after-snap-refresh-{type}-{name}-to-stable-rev job does not exist anymore, so the snapd/snap-verify-after-refresh-{type}-{name}-to-stable-rev should depend on the refresh job itself (which was modified in the aforementioned PR to do the reboot itself).

Resolved issues

An issue that showed up while testing a customer project:

https://certification.canonical.com/hardware/202302-31257/submission/393904/test-results/?term=cascade-kernel-to-base-rev

See discussion on MM for more information.

Documentation

Tests

Running checkbox-cli list-bootstrapped com.canonical.certification::snap-refresh-revert on my laptop yield the following results:

Before this patch:

com.canonical.certification::snap_revision_info
com.canonical.certification::snapd/snap-refresh-snapd-snapd-to-base-rev
com.canonical.certification::snapd/log-attach-after-snap-refresh-snapd-snapd-to-base-rev
com.canonical.certification::snapd/snap-verify-after-refresh-snapd-snapd-to-base-rev
com.canonical.certification::snapd/snap-revert-snapd-snapd-from-base-rev
com.canonical.certification::snapd/log-attach-after-snap-revert-snapd-snapd-from-base-rev
com.canonical.certification::snapd/snap-verify-after-revert-snapd-snapd-from-base-rev
com.canonical.certification::snapd/snap-refresh-snapd-snapd-to-stable-rev
com.canonical.certification::snapd/log-attach-after-snap-refresh-snapd-snapd-to-stable-rev

After this patch:

com.canonical.certification::snap_revision_info
com.canonical.certification::snapd/snap-refresh-snapd-snapd-to-base-rev
com.canonical.certification::snapd/log-attach-after-snap-refresh-snapd-snapd-to-base-rev
com.canonical.certification::snapd/snap-verify-after-refresh-snapd-snapd-to-base-rev
com.canonical.certification::snapd/snap-revert-snapd-snapd-from-base-rev
com.canonical.certification::snapd/log-attach-after-snap-revert-snapd-snapd-from-base-rev
com.canonical.certification::snapd/snap-verify-after-revert-snapd-snapd-from-base-rev
com.canonical.certification::snapd/snap-refresh-snapd-snapd-to-stable-rev
com.canonical.certification::snapd/log-attach-after-snap-refresh-snapd-snapd-to-stable-rev
com.canonical.certification::snapd/snap-verify-after-refresh-snapd-snapd-to-stable-rev
com.canonical.certification::snapd/snap-revert-snapd-snapd-from-stable-rev
com.canonical.certification::snapd/log-attach-after-snap-revert-snapd-snapd-from-stable-rev
com.canonical.certification::snapd/snap-verify-after-revert-snapd-snapd-from-stable-rev

(the last 4 jobs are now present that are verifying the snap after an update to stable channel, and then running extra revert and verification steps)

codecov[bot] commented 1 month ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 47.56%. Comparing base (2605982) to head (f939731). Report is 1 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1519 +/- ## ======================================= Coverage 47.56% 47.56% ======================================= Files 369 369 Lines 39596 39596 Branches 6691 6691 ======================================= Hits 18833 18833 Misses 20051 20051 Partials 712 712 ``` | [Flag](https://app.codecov.io/gh/canonical/checkbox/pull/1519/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | Coverage Δ | | |---|---|---| | [provider-base](https://app.codecov.io/gh/canonical/checkbox/pull/1519/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | `24.04% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.