canonical / checkbox

Checkbox is a testing framework used to validate device compatibility with Ubuntu Linux. It’s the testing tool developed for the purposes of the Ubuntu Certification program.
https://checkbox.readthedocs.io
GNU General Public License v3.0
33 stars 50 forks source link

Use the retry decorator in WiFi tests (Bugfix) #1488

Closed pieqq closed 1 month ago

pieqq commented 2 months ago

Description

The main raisons d'être of this PR are:

Resolved issues

CHECKBOX-1543

Documentation

Nothing is modified in terms of Checkbox jobs or external-facing documentation.

Tests

codecov[bot] commented 2 months ago

Codecov Report

Attention: Patch coverage is 91.07143% with 5 lines in your changes missing coverage. Please review.

Project coverage is 47.50%. Comparing base (104c931) to head (e41f30d). Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
providers/base/bin/wifi_nmcli_test.py 90.74% 4 Missing and 1 partial :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1488 +/- ## ========================================== + Coverage 47.48% 47.50% +0.01% ========================================== Files 369 369 Lines 39586 39549 -37 Branches 6685 6678 -7 ========================================== - Hits 18798 18788 -10 + Misses 20077 20050 -27 Partials 711 711 ``` | [Flag](https://app.codecov.io/gh/canonical/checkbox/pull/1488/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | Coverage Δ | | |---|---|---| | [checkbox-support](https://app.codecov.io/gh/canonical/checkbox/pull/1488/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | `60.49% <100.00%> (ø)` | | | [provider-base](https://app.codecov.io/gh/canonical/checkbox/pull/1488/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | `24.02% <90.74%> (-0.01%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Hook25 commented 2 months ago

Testing this on 3 devices in the lab (focal, jammy, noble): https://github.com/canonical/checkbox/actions/runs/10958861773

For reference, here is the command:

gh workflow run dispatch_lab_job.yaml --ref 1543-retry-decorator-wifi-tests -f 'matrix_to_create=[{"data_source":"distro: desktop-24-04-uefi","queue":"201912-27623","match":".*wireless.*","test_plan":"com.canonical.certification::sru"},{"data_source":"distro: desktop-22-04-2-uefi","queue":"202002-27717","match":".*wireless.*","test_plan":"com.canonical.certification::sru"},{"data_source":"distro: desktop-20-04-1-uefi","queue":"201904-26941","match":".*wireless.*","test_plan":"com.canonical.certification::sru"}]'