canonical / checkbox

Checkbox is a testing framework used to validate device compatibility with Ubuntu Linux. It’s the testing tool developed for the purposes of the Ubuntu Certification program.

GNU General Public License v3.0

33 stars 49 forks source link

Description

Some of the machines in linux-generic 6.8 and linux-generic-hwe-20.04 pool failed at suspend-time-check without any log (see #1547) This issue occurred because we have several different names for the suspend log files depending on the suspend test we are running.

In this case, the 1_suspend_single_times.log was not created for devices with only integrated GPU, and suspend/1_suspend-time-check_{product}_auto test was failing because of this.

We have included a change to use the log created by suspend/suspend_advanced_auto which is named suspend_single_times.log. This is still confusing but solves the current issue. We will try to improve the structure of these test in a follow-up PR.

Similar changes have been implemented for suspend/{{ index }}_suspend-single-log-attach_{{ product_slug }}_auto

Resolved issues

1547

CHECKBOX-1619

Documentation

Tests

This test run shows the test passing on the failing device: https://certification.canonical.com/hardware/202008-28166/submission/401708/test-results/?term=suspend-time

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 48.03%. Comparing base (b2822b6) to head (ca1529b). Report is 4 commits behind head on main.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #1597 +/- ## ======================================= Coverage 48.03% 48.03% ======================================= Files 371 371 Lines 39850 39850 Branches 6734 6734 ======================================= Hits 19140 19140 Misses 19993 19993 Partials 717 717 ``` | [Flag](https://app.codecov.io/gh/canonical/checkbox/pull/1597/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | Coverage Δ | | |---|---|---| | [provider-base](https://app.codecov.io/gh/canonical/checkbox/pull/1597/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | `24.71% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical#carryforward-flags-in-the-pull-request-comment) to find out more.