canonical / checkbox

Checkbox is a testing framework used to validate device compatibility with Ubuntu Linux. It’s the testing tool developed for the purposes of the Ubuntu Certification program.
https://checkbox.readthedocs.io
GNU General Public License v3.0
33 stars 50 forks source link

Fix touchscreen tests on ARM64 via remote (bugfix) #1528

Closed Hook25 closed 1 month ago

Hook25 commented 1 month ago

Description

When running the touchscreen tests via remote, the user is promted to rotate the rectangle on a window but the user-interact-verify job (silently) fails to start the window at all. When the stderr is un-silenced, the following error is printed in the console:

file:///snap/checkboxXX/[...]/touch_zoom_test.qml:74 Timer is not a type

Timer is a builtin type of QT and this happens for any other type (removing the timer just makes it fail on the next builtin type).

After much investigation, and thanks to the reported fact that the test starts fine if launched from the checkbox.shell or locally, I found that the problem is the missing envvar in this PR. This fixes the propagation of the envvar. I have whitelisted it instead of explicitly requiring it in the test because I believe that one doesn't have to understand exactly how QT works when writing a test, so they don't need to reason about which QT envvars are needed.

Resolved issues

Fixes: CHECKBOX-1511 Fixes: https://github.com/canonical/checkbox/issues/1371

Documentation

N/A

Tests

I have built and ran the test via remote with the following launcher. The window now shows up even via remote when before it did not.

    #!/usr/bin/env checkbox-cli
    [launcher]
    launcher_version = 1
    stock_reports = text
    [test plan]
    unit = com.canonical.certification::touchscreen-cert-manual
    forced = yes
    [test selection]
    forced=yes
    exclude=com.canonical.certification::touchscreen/drag-n-drop
    match = com.canonical.certification::touchscreen/multitouch-rotate
    [manifest]
    com.canonical.certification::has_touchscreen=yes

To test it yourself use this checkbox24 snap, built from this branch (or build it yourself on arm64):

  1. Install the snap
  2. Install the frontend
  3. Run remote with the above launcher
  4. Press Return
  5. A window with a blue rectangle should show up on the DUT

Note: This is a core24 runtime snap. Remember CheckboxXX core only work on the XX version (so this doesn't work on jammy, only noble), if you want to test it on jammy I can build a core22 snap for you, just let me know

Hook25 commented 1 month ago

Proof of this working on the original hardware in the lab: image

codecov[bot] commented 1 month ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 47.56%. Comparing base (17ceef1) to head (5a432d2). Report is 10 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1528 +/- ## ======================================= Coverage 47.56% 47.56% ======================================= Files 369 369 Lines 39596 39596 Branches 6691 6691 ======================================= Hits 18833 18833 Misses 20051 20051 Partials 712 712 ``` | [Flag](https://app.codecov.io/gh/canonical/checkbox/pull/1528/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | Coverage Δ | | |---|---|---| | [checkbox-ng](https://app.codecov.io/gh/canonical/checkbox/pull/1528/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | `68.14% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.