canonical / checkbox

Checkbox is a testing framework used to validate device compatibility with Ubuntu Linux. It’s the testing tool developed for the purposes of the Ubuntu Certification program.
https://checkbox.readthedocs.io
GNU General Public License v3.0
33 stars 50 forks source link

Use wwan-automated nested part in SRU test plan (bugfix) #1499

Closed pieqq closed 1 month ago

pieqq commented 1 month ago

WARNING: This modifies com.canonical.certification::sru-server

Description

Replacemobilebroadband-cert-automated with wwan-automated nested part, as the wwan-automated test plan includes better coverage for WWAN-related tests.

Add after-suspend-wwan-automated counterpart to make sure there are no issues with WWAN after resuming from suspend.

This is a follow-up to work done by QA in the certification-client desktop test plans (see https://github.com/canonical/checkbox/pull/821)

IMPORTANT: PR #1500 is required before merging this.

Resolved issues

Fix CER-2738

Documentation

Tests

Used checkbox-cli expand com.canonical.certification::sru to generate the list of executed jobs and templates before and after applying this patch. The diff is the following:

$ diff /var/tmp/cer-2738-before.log /var/tmp/cer-2738-after.log 
2a3
> Job 'com.canonical.certification::after-suspend-wwan/detect'
74,75d74
< Job 'com.canonical.certification::mobilebroadband/cdma_connection'
< Job 'com.canonical.certification::mobilebroadband/gsm_connection'
140a140,143
> Template 'com.canonical.certification::wwan/3gpp-scan-manufacturer-model-hw_id-auto'
> Template 'com.canonical.certification::wwan/check-sim-present-manufacturer-model-hw_id-auto'
> Job 'com.canonical.certification::wwan/detect'
> Template 'com.canonical.certification::wwan/gsm-connection-manufacturer-model-hw_id-auto'

showing that:

codecov[bot] commented 1 month ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 47.56%. Comparing base (ba4bbb7) to head (a4dfe62). Report is 3 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1499 +/- ## ======================================= Coverage 47.56% 47.56% ======================================= Files 369 369 Lines 39596 39596 Branches 6691 6691 ======================================= Hits 18833 18833 Misses 20051 20051 Partials 712 712 ``` | [Flag](https://app.codecov.io/gh/canonical/checkbox/pull/1499/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | Coverage Δ | | |---|---|---| | [provider-sru](https://app.codecov.io/gh/canonical/checkbox/pull/1499/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | `97.97% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.