canonical / checkbox

Checkbox is a testing framework used to validate device compatibility with Ubuntu Linux. It’s the testing tool developed for the purposes of the Ubuntu Certification program.
https://checkbox.readthedocs.io
GNU General Public License v3.0
34 stars 50 forks source link

Ordering of storage tests (BugFix) #1454

Closed fernando79513 closed 2 months ago

fernando79513 commented 2 months ago

WARNING: This modifies com.canonical.certification::sru-server

Description

As commented in #910, if the USB manual tests fail, when user reruns the failed jobs, the execution order will lose the original sequence.

This PR aims to solve this issue by merging the 3 storage tests insertion/storage/removal into one unique test that performs the three actions.

Here you can find the conversation in Mattermost

Resolved issues

91

https://warthogs.atlassian.net/browse/CHECKBOX-1497

Documentation

N/A

Tests

Storage tests (22.04 laptop) submission_2024-09-16T10.44.35.228920.zip Zapper USB tests submission_2024-09-16T11.58.24.029262.zip

codecov[bot] commented 2 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 47.11%. Comparing base (3ba7e2c) to head (550039b). Report is 2 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1454 +/- ## ========================================== + Coverage 47.10% 47.11% +0.01% ========================================== Files 368 368 Lines 39435 39443 +8 Branches 6667 6668 +1 ========================================== + Hits 18574 18585 +11 + Misses 20156 20155 -1 + Partials 705 703 -2 ``` | [Flag](https://app.codecov.io/gh/canonical/checkbox/pull/1454/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | Coverage Δ | | |---|---|---| | [checkbox-support](https://app.codecov.io/gh/canonical/checkbox/pull/1454/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | `60.49% <100.00%> (+0.11%)` | :arrow_up: | | [provider-base](https://app.codecov.io/gh/canonical/checkbox/pull/1454/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | `23.04% <ø> (ø)` | | | [provider-certification-client](https://app.codecov.io/gh/canonical/checkbox/pull/1454/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | `57.14% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

baconYao commented 1 month ago

Verify Pass on Boashan G1200

snap info

checkbox22                 4.2.0-dev103      1207   latest/edge      ce-certification-qa  -

Submission: https://certification.canonical.com/hardware/202307-31864/submission/398134/test-results/?term=zapper-usb