canonical / checkbox

Checkbox is a testing framework used to validate device compatibility with Ubuntu Linux. It’s the testing tool developed for the purposes of the Ubuntu Certification program.
https://checkbox.readthedocs.io
GNU General Public License v3.0
33 stars 49 forks source link

Remove gtk from camera multiple resolutions (New) #1400

Closed fernando79513 closed 2 months ago

fernando79513 commented 3 months ago

Description

The main objective of this test was to remove Clutter and Gtk dependencies from camera_test.py so we could run the multipe-resolution-images job on core devices without a display. The testing on the available core devices with camera required some additional fixes to the script, So this PR contains a bigger series of changes.:

Resolved issues

https://warthogs.atlassian.net/browse/CHECKBOX-1200

Documentation

The camera test script has been documented further inside the code adding more clear comments

Tests

To test this new script on a device, you can run any of the automated or manual camera tests:

checkbox-cli run com.canonical.certification::camera-cert-automated
checkbox-cli run com.canonical.certification::camera-cert-manual

submission_2024-08-08T14.05.34.803516.zip

Note: On shiner-imx8-med-pdk001, gstreamer is not able to capture the image at 1640x1232, so the test times out. If that resolution is excluded, the test passes.

==============[ Running job 3 / 7. Estimated time left: 0:00:25 ]===============
-----------[ Webcam multiple resolution capture test for i.MX6S_CSI ]-----------
ID: com.canonical.certification::camera/multiple-resolution-images_video0
Category: com.canonical.plainbox::camera
... 8< -------------------------------------------------------------------------
Failed to validate image /tmp/camera_test_RG101640x1232tuwu9jue.jpg
Format: RG10 (10-bit Bayer RGRG/GBGB)
Resolutions: 3280x2464,1920x1080,1640x1232,640x480
Format: GBRG (8-bit Bayer GBGB/RGRG)
Resolutions: 3280x2464,1920x1080,1640x1232,640x480

Taking multiple images using the RG10 pixelformat
Saving debug image to /var/tmp/checkbox-ng/sessions/checkbox-run-2024-08-08T06.32.44.session/session-share/resolution_test_image_video0.jpg
Failed to capture image with fswebcam, using gstreamer
Timeout exceeded
Image saved to /var/tmp/checkbox-ng/sessions/checkbox-run-2024-08-08T06.32.44.session/session-share/resolution_test_image_video0.jpg
Taking a picture at 640x480
Failed to capture image with fswebcam, using gstreamer
Image saved to /tmp/camera_test_RG10640x4806wjpgc2m.jpg
Validated image /tmp/camera_test_RG10640x4806wjpgc2m.jpg
Taking a picture at 1640x1232
Failed to capture image with fswebcam, using gstreamer
Timeout exceeded
Image saved to /tmp/camera_test_RG101640x1232tuwu9jue.jpg
Image dimensions not found in JPEG file
------------------------------------------------------------------------- >8 ---
Outcome: job failed

With 1640x1232 excluded:

==============[ Running job 3 / 7. Estimated time left: 0:00:25 ]===============
-----------[ Webcam multiple resolution capture test for i.MX6S_CSI ]-----------
ID: com.canonical.certification::camera/multiple-resolution-images_video0
Category: com.canonical.plainbox::camera
... 8< -------------------------------------------------------------------------
Format: RG10 (10-bit Bayer RGRG/GBGB)
Resolutions: 3280x2464,1920x1080,1640x1232,640x480
Format: GBRG (8-bit Bayer GBGB/RGRG)
Resolutions: 3280x2464,1920x1080,1640x1232,640x480

Taking multiple images using the RG10 pixelformat
Saving debug image to /var/tmp/checkbox-ng/sessions/checkbox-run-2024-08-08T06.39.35.session/session-share/resolution_test_image_video0.jpg
Failed to capture image with fswebcam, using gstreamer
Image saved to /var/tmp/checkbox-ng/sessions/checkbox-run-2024-08-08T06.39.35.session/session-share/resolution_test_image_video0.jpg
Taking a picture at 640x480
Failed to capture image with fswebcam, using gstreamer
Image saved to /tmp/camera_test_RG10640x480va3xlq6e.jpg
Validated image /tmp/camera_test_RG10640x480va3xlq6e.jpg
Taking a picture at 1920x1080
Failed to capture image with fswebcam, using gstreamer
Image saved to /tmp/camera_test_RG101920x1080sg0kuplm.jpg
Validated image /tmp/camera_test_RG101920x1080sg0kuplm.jpg
Taking a picture at 3280x2464
Failed to capture image with fswebcam, using gstreamer
Image saved to /tmp/camera_test_RG103280x2464gvjpp57s.jpg
Validated image /tmp/camera_test_RG103280x2464gvjpp57s.jpg
------------------------------------------------------------------------- >8 ---
Outcome: job passed
codecov[bot] commented 3 months ago

Codecov Report

Attention: Patch coverage is 91.19171% with 17 lines in your changes missing coverage. Please review.

Project coverage is 47.29%. Comparing base (0881101) to head (1ca94f1). Report is 15 commits behind head on main.

Files with missing lines Patch % Lines
providers/base/bin/camera_test.py 91.19% 12 Missing and 5 partials :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1400 +/- ## ========================================== + Coverage 45.87% 47.29% +1.42% ========================================== Files 367 367 Lines 39156 39490 +334 Branches 6622 6671 +49 ========================================== + Hits 17961 18678 +717 + Misses 20504 20099 -405 - Partials 691 713 +22 ``` | [Flag](https://app.codecov.io/gh/canonical/checkbox/pull/1400/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | Coverage Δ | | |---|---|---| | [provider-base](https://app.codecov.io/gh/canonical/checkbox/pull/1400/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | `24.02% <91.19%> (+4.11%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.