canonical / checkbox

Checkbox is a testing framework used to validate device compatibility with Ubuntu Linux. It’s the testing tool developed for the purposes of the Ubuntu Certification program.
https://checkbox.readthedocs.io
GNU General Public License v3.0
33 stars 50 forks source link

Add the wake on lan manual test to the desktop test plan per customer's request (Bugfix) #1455

Closed eugene-yujinwu closed 1 month ago

eugene-yujinwu commented 2 months ago

Description

Add the wake on lan tests to the desktop test plan per the feature request of the desktop team of Sutton. Add a manifest has_ethernet_wake_on_lan_suppor to let the tester to decide if the platform support or not of WOL.

Resolved issues

https://bugs.launchpad.net/sutton/+bug/2072930

Documentation

Tests

$checkbox-cli list-bootstrapped com.canonical.certification::client-cert-desktop-24-04-manual

Skipped file: /usr/share/checkbox-provider-base/units/camera/README.rst Skipped file: /usr/share/checkbox-provider-base/units/snapd/README.md Skipped file: /usr/share/checkbox-provider-base/units/stress/suspend_cycles_reboot.md com.canonical.certification::miscellanea/device_check com.canonical.certification::cdimage com.canonical.certification::dmi_present com.canonical.certification::dmi com.canonical.certification::lsb com.canonical.certification::modprobe_json com.canonical.certification::dkms_info_json com.canonical.certification::meminfo com.canonical.certification::module com.canonical.certification::dmi_attachment com.canonical.certification::environment com.canonical.certification::requirements com.canonical.certification::cpuinfo com.canonical.certification::snap com.canonical.certification::dpkg com.canonical.certification::udev_attachment com.canonical.certification::kernel_cmdline_attachment com.canonical.certification::uname com.canonical.certification::sysfs_attachment com.canonical.certification::device com.canonical.certification::efi com.canonical.certification::package com.canonical.certification::executable com.canonical.certification::lsblk_attachment com.canonical.certification::lspci_standard_config_json com.canonical.certification::udev_json com.canonical.certification::system_info_json com.canonical.certification::raw_devices_dmi_json com.canonical.certification::miscellanea/submission-resources com.canonical.certification::info/systemd-analyze com.canonical.certification::info/systemd-analyze-critical-chain com.canonical.plainbox::manifest com.canonical.certification::audio/speaker-headphone-plug-detection com.canonical.certification::audio/microphone-plug-detection com.canonical.certification::audio/list_devices com.canonical.certification::audio/playback_headphones com.canonical.certification::audio/alsa_record_playback_external com.canonical.certification::audio/playback_auto com.canonical.certification::audio/alsa_record_playback_internal com.canonical.certification::audio/channels com.canonical.certification::audio/external-linein com.canonical.certification::audio/external-lineout com.canonical.certification::bluetooth/detect-output com.canonical.certification::bluetooth4/HOGP-mouse com.canonical.certification::bluetooth4/HOGP-keyboard com.canonical.certification::bluetooth/audio-a2dp com.canonical.certification::bluetooth/audio_record_playback com.canonical.certification::camera/detect com.canonical.certification::camera/still_video0 com.canonical.certification::camera/display_video0 com.canonical.certification::thunderbolt3/insert com.canonical.certification::thunderbolt3/storage-test com.canonical.certification::thunderbolt3/remove com.canonical.certification::monitor/1_powersaving_CometLake-U_GT2UHDGraphics com.canonical.certification::power-management/light_sensor com.canonical.certification::monitor/1_dim_brightness_CometLake-U_GT2__UHDGraphics com.canonical.certification::monitor/1_displayport_CometLake-U_GT2UHDGraphics com.canonical.certification::audio/1_playback_displayport_CometLake-U_GT2UHDGraphics com.canonical.certification::monitor/1_type-c_displayport_CometLake-U_GT2UHDGraphics com.canonical.certification::audio/1_playback_type-c_displayport_CometLake-U_GT2UHDGraphics com.canonical.certification::monitor/1_type-c_hdmi_CometLake-U_GT2UHDGraphics com.canonical.certification::audio/1_playback_type-c_hdmi_CometLake-U_GT2UHDGraphics com.canonical.certification::monitor/1_type-c_vga_CometLake-U_GT2UHDGraphics com.canonical.certification::monitor/1_dvi_CometLake-U_GT2UHDGraphics com.canonical.certification::monitor/1_hdmi_CometLake-U_GT2UHDGraphics com.canonical.certification::audio/1_playback_hdmi_CometLake-U_GT2UHDGraphics com.canonical.certification::monitor/1_thunderbolt3_CometLake-U_GT2UHDGraphics com.canonical.certification::audio/1_playback_thunderbolt3_CometLake-U_GT2UHDGraphics com.canonical.certification::thunderbolt3/daisy-chain com.canonical.certification::monitor/multi-head com.canonical.certification::miscellanea/chvt com.canonical.certification::graphics/1_maximum_resolution_CometLake-U_GT2UHDGraphics com.canonical.certification::graphics/1_rotation_CometLake-U_GT2UHDGraphics com.canonical.certification::graphics/1_video_CometLake-U_GT2UHDGraphics com.canonical.certification::graphics/VESA_drivers_not_in_use com.canonical.certification::graphics/1_cycle_resolution_CometLake-U_GT2UHDGraphics com.canonical.certification::graphics/1_valid_glxgears_CometLake-U_GT2__UHDGraphics com.canonical.certification::graphics/1_valid_glxgears_fullscreen_CometLake-U_GT2UHDGraphics com.canonical.certification::graphics/2_valid_glxgears_GP108GLMQuadroP520 com.canonical.certification::graphics/2_valid_glxgears_fullscreen_GP108GLMQuadroP520 com.canonical.certification::graphics/valid_glxgears com.canonical.certification::graphics/valid_glxgears_fullscreen com.canonical.certification::input/accelerometer com.canonical.certification::input/pointing_Lenovo_Essential_Wireless_Keyboard_and_Mouse_Combo_MOUSE_1 com.canonical.certification::input/pointing_ELAN_Touchscreen_TOUCHSCREEN_2 com.canonical.certification::input/pointing_SynPS_2_Synaptics_TouchPad_TOUCHPAD_3 com.canonical.certification::input/pointing_TPPS_2_Elan_TrackPoint_MOUSE_4 com.canonical.certification::input/clicking_Lenovo_Essential_Wireless_Keyboard_and_Mouse_Combo_MOUSE_1 com.canonical.certification::input/clicking_SynPS_2_Synaptics_TouchPad_TOUCHPAD_2 com.canonical.certification::input/clicking_TPPS_2_Elan_TrackPoint_MOUSE_3 com.canonical.certification::input/keyboard com.canonical.certification::disk/detect com.canonical.certification::disk_resource com.canonical.certification::disk/hdd-parking com.canonical.certification::fingerprint/detect com.canonical.certification::fingerprint/enroll com.canonical.certification::fingerprint/verify-no-match com.canonical.certification::fingerprint/verify-match com.canonical.certification::fingerprint/unlock com.canonical.certification::fingerprint/delete com.canonical.certification::keys/lock-screen com.canonical.certification::keys/super com.canonical.certification::keys/brightness com.canonical.certification::keys/media-control com.canonical.certification::keys/mute com.canonical.certification::keys/volume com.canonical.certification::keys/video-out com.canonical.certification::keys/wireless com.canonical.certification::keys/keyboard-backlight com.canonical.certification::keys/microphone-mute com.canonical.certification::keys/power-button com.canonical.certification::keys/power-button-event com.canonical.certification::keys/fn-lock com.canonical.certification::camera/led_video0 com.canonical.certification::led/caps-lock com.canonical.certification::led/numeric-keypad com.canonical.certification::led/power com.canonical.certification::led/fn com.canonical.certification::led/mute com.canonical.certification::led/microphone-mute com.canonical.certification::led/wireless com.canonical.certification::mediacard/sdhc-insert com.canonical.certification::mediacard/sdhc-storage com.canonical.certification::mediacard/sdhc-remove com.canonical.certification::ethernet/detect com.canonical.certification::ethernet/hotplug-enp0s31f6 com.canonical.certification::networking/info_device1_enp0s31f6 com.canonical.certification::optical/detect com.canonical.certification::touchpad/basic com.canonical.certification::touchpad/palm-rejection com.canonical.certification::touchpad/continuous-move com.canonical.certification::touchpad/singletouch-selection com.canonical.certification::touchpad/drag-and-drop com.canonical.certification::touchpad/multitouch-rightclick com.canonical.certification::touchpad/multitouch com.canonical.certification::touchscreen/drag-n-drop com.canonical.certification::touchscreen/multitouch-zoom com.canonical.certification::touchscreen/multitouch-rotate com.canonical.certification::touchscreen/evdev/single-touch-tap-ELAN_Touchscreen com.canonical.certification::touchscreen/evdev/2-touch-tap-ELAN_Touchscreen com.canonical.certification::touchscreen/evdev/3-touch-tap-ELAN_Touchscreen com.canonical.certification::touchscreen/evdev/4-touch-tap-ELAN_Touchscreen com.canonical.certification::usb/detect com.canonical.certification::usb/HID com.canonical.certification::usb/insert com.canonical.certification::usb/storage-automated com.canonical.certification::usb/remove com.canonical.certification::usb com.canonical.certification::usb3/insert com.canonical.certification::usb3/storage-automated com.canonical.certification::usb3/remove com.canonical.certification::usb-c/c-to-a-adapter/hid com.canonical.certification::usb-c/c-to-a-adapter/insert com.canonical.certification::usb-c/c-to-a-adapter/storage-automated com.canonical.certification::usb-c/c-to-a-adapter/remove com.canonical.certification::usb-c/hid com.canonical.certification::usb-c/insert com.canonical.certification::usb-c/storage-automated com.canonical.certification::usb-c/remove com.canonical.certification::usb-c/c-to-ethernet-adapter-insert com.canonical.certification::power-management/lid_close_suspend_open com.canonical.certification::power-management/lid com.canonical.certification::rtc com.canonical.certification::sleep com.canonical.certification::keys/sleep com.canonical.certification::suspend/suspend_advanced_auto com.canonical.certification::after-suspend-miscellanea/chvt com.canonical.certification::suspend/display_after_suspend com.canonical.certification::after-suspend-graphics/1_maximum_resolution_CometLake-U_GT2UHDGraphics com.canonical.certification::after-suspend-graphics/1_rotation_CometLake-U_GT2__UHDGraphics com.canonical.certification::after-suspend-graphics/1_video_CometLake-U_GT2UHDGraphics com.canonical.certification::after-suspend-graphics/1_cycle_resolution_CometLake-U_GT2UHDGraphics com.canonical.certification::after-suspend-graphics/1_valid_glxgears_CometLake-U_GT2UHDGraphics com.canonical.certification::after-suspend-graphics/1_valid_glxgears_fullscreen_CometLake-U_GT2UHDGraphics com.canonical.certification::after-suspend-graphics/2_valid_glxgears_GP108GLMQuadroP520 com.canonical.certification::after-suspend-graphics/2_valid_glxgears_fullscreen_GP108GLMQuadroP520 com.canonical.certification::after-suspend-graphics/valid_glxgears com.canonical.certification::after-suspend-graphics/valid_glxgears_fullscreen com.canonical.certification::suspend/1_xrandr_screens_after_suspend.tar.gz_auto com.canonical.certification::after-suspend-monitor/1_powersaving_CometLake-U_GT2UHDGraphics com.canonical.certification::after-suspend-power-management/light_sensor com.canonical.certification::after-suspend-monitor/1_dim_brightness_CometLake-U_GT2UHDGraphics com.canonical.certification::after-suspend-monitor/1_displayport_CometLake-U_GT2UHDGraphics com.canonical.certification::after-suspend-audio/1_playback_displayport_CometLake-U_GT2UHDGraphics com.canonical.certification::after-suspend-monitor/1_type-c_displayport_CometLake-U_GT2__UHDGraphics com.canonical.certification::after-suspend-audio/1_playback_type-c_displayport_CometLake-U_GT2UHDGraphics com.canonical.certification::after-suspend-monitor/1_type-c_hdmi_CometLake-U_GT2UHDGraphics com.canonical.certification::after-suspend-audio/1_playback_type-c_hdmi_CometLake-U_GT2UHDGraphics com.canonical.certification::after-suspend-monitor/1_type-c_vga_CometLake-U_GT2UHDGraphics com.canonical.certification::after-suspend-monitor/1_dvi_CometLake-U_GT2UHDGraphics com.canonical.certification::after-suspend-monitor/1_hdmi_CometLake-U_GT2UHDGraphics com.canonical.certification::after-suspend-audio/1_playback_hdmi_CometLake-U_GT2UHDGraphics com.canonical.certification::after-suspend-monitor/1_thunderbolt3_CometLake-U_GT2UHDGraphics com.canonical.certification::after-suspend-audio/1_playback_thunderbolt3_CometLake-U_GT2UHDGraphics com.canonical.certification::after-suspend-thunderbolt3/daisy-chain com.canonical.certification::after-suspend-monitor/multi-head com.canonical.certification::suspend/oops_after_suspend com.canonical.certification::suspend/oops_results_after_suspend.log com.canonical.certification::led/power-blink-suspend com.canonical.certification::led/suspend com.canonical.certification::after-suspend-audio/speaker-headphone-plug-detection com.canonical.certification::after-suspend-audio/microphone-plug-detection com.canonical.certification::after-suspend-audio/playback_headphones com.canonical.certification::after-suspend-audio/alsa_record_playback_external com.canonical.certification::after-suspend-audio/playback_auto com.canonical.certification::after-suspend-audio/alsa_record_playback_internal com.canonical.certification::after-suspend-audio/channels com.canonical.certification::after-suspend-audio/external-linein com.canonical.certification::after-suspend-audio/external-lineout com.canonical.certification::after-suspend-bluetooth4/HOGP-mouse com.canonical.certification::after-suspend-bluetooth4/HOGP-keyboard com.canonical.certification::after-suspend-bluetooth/audio-a2dp com.canonical.certification::after-suspend-bluetooth/audio_record_playback com.canonical.certification::after-suspend-camera/still_video0 com.canonical.certification::after-suspend-camera/display_video0 com.canonical.certification::after-suspend-thunderbolt3/insert com.canonical.certification::after-suspend-thunderbolt3/storage-test com.canonical.certification::after-suspend-thunderbolt3/remove com.canonical.certification::after-suspend-input/accelerometer com.canonical.certification::after-suspend-input/pointing_Lenovo_Essential_Wireless_Keyboard_and_Mouse_Combo_MOUSE_1 com.canonical.certification::after-suspend-input/pointing_ELAN_Touchscreen_TOUCHSCREEN_2 com.canonical.certification::after-suspend-input/pointing_SynPS_2_Synaptics_TouchPad_TOUCHPAD_3 com.canonical.certification::after-suspend-input/pointing_TPPS_2_Elan_TrackPoint_MOUSE_4 com.canonical.certification::after-suspend-input/keyboard com.canonical.certification::after-suspend-fingerprint/detect com.canonical.certification::after-suspend-fingerprint/enroll com.canonical.certification::after-suspend-fingerprint/verify-no-match com.canonical.certification::after-suspend-fingerprint/verify-match com.canonical.certification::after-suspend-fingerprint/unlock com.canonical.certification::after-suspend-fingerprint/delete com.canonical.certification::after-suspend-keys/lock-screen com.canonical.certification::after-suspend-keys/super com.canonical.certification::after-suspend-keys/brightness com.canonical.certification::after-suspend-keys/media-control com.canonical.certification::after-suspend-keys/mute com.canonical.certification::after-suspend-keys/volume com.canonical.certification::after-suspend-keys/video-out com.canonical.certification::after-suspend-keys/wireless com.canonical.certification::after-suspend-keys/keyboard-backlight com.canonical.certification::after-suspend-keys/microphone-mute com.canonical.certification::after-suspend-keys/power-button com.canonical.certification::after-suspend-keys/power-button-event com.canonical.certification::after-suspend-keys/fn-lock com.canonical.certification::after-suspend-camera/led_video0 com.canonical.certification::after-suspend-led/caps-lock com.canonical.certification::after-suspend-led/numeric-keypad com.canonical.certification::after-suspend-led/power com.canonical.certification::after-suspend-led/mute com.canonical.certification::after-suspend-led/microphone-mute com.canonical.certification::after-suspend-led/fn com.canonical.certification::after-suspend-mediacard/sdhc-insert com.canonical.certification::after-suspend-mediacard/sdhc-storage com.canonical.certification::after-suspend-mediacard/sdhc-remove com.canonical.certification::after-suspend-ethernet/detect com.canonical.certification::after-suspend-ethernet/hotplug-enp0s31f6 com.canonical.certification::after-suspend-networking/info_device1_enp0s31f6 com.canonical.certification::after-suspend-touchpad/basic com.canonical.certification::after-suspend-touchpad/palm-rejection com.canonical.certification::after-suspend-touchpad/continuous-move com.canonical.certification::after-suspend-touchpad/singletouch-selection com.canonical.certification::after-suspend-touchpad/drag-and-drop com.canonical.certification::after-suspend-touchpad/multitouch-rightclick com.canonical.certification::after-suspend-touchpad/multitouch com.canonical.certification::after-suspend-touchscreen/drag-n-drop com.canonical.certification::after-suspend-touchscreen/multitouch-zoom com.canonical.certification::after-suspend-touchscreen/multitouch-rotate com.canonical.certification::after-suspend-touchscreen/evdev/single-touch-tap-ELAN_Touchscreen com.canonical.certification::after-suspend-touchscreen/evdev/2-touch-tap-ELAN_Touchscreen com.canonical.certification::after-suspend-touchscreen/evdev/3-touch-tap-ELAN_Touchscreen com.canonical.certification::after-suspend-touchscreen/evdev/4-touch-tap-ELAN_Touchscreen com.canonical.certification::after-suspend-usb/HID com.canonical.certification::after-suspend-usb/insert com.canonical.certification::after-suspend-usb/storage-automated com.canonical.certification::after-suspend-usb/remove com.canonical.certification::after-suspend-usb3/insert com.canonical.certification::after-suspend-usb3/storage-automated com.canonical.certification::after-suspend-usb3/remove com.canonical.certification::after-suspend-usb-c/c-to-a-adapter/hid com.canonical.certification::after-suspend-usb-c/c-to-a-adapter/insert com.canonical.certification::after-suspend-usb-c/c-to-a-adapter/storage-automated com.canonical.certification::after-suspend-usb-c/c-to-a-adapter/remove com.canonical.certification::after-suspend-usb-c/hid com.canonical.certification::after-suspend-usb-c/insert com.canonical.certification::after-suspend-usb-c/storage-automated com.canonical.certification::after-suspend-usb-c/remove com.canonical.certification::after-suspend-usb-c/c-to-ethernet-adapter-insert com.canonical.certification::ethernet/wol_S5_enp0s31f6 com.canonical.certification::ethernet/wol_S3_enp0s31f6

![Uploading Screenshot from 2024-09-05 15-34-44.png…]()

codecov[bot] commented 2 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 47.55%. Comparing base (0d0f721) to head (e24e44b). Report is 1 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1455 +/- ## ======================================= Coverage 47.55% 47.55% ======================================= Files 369 369 Lines 39585 39585 Branches 6685 6685 ======================================= Hits 18824 18824 Misses 20050 20050 Partials 711 711 ``` | [Flag](https://app.codecov.io/gh/canonical/checkbox/pull/1455/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | Coverage Δ | | |---|---|---| | [provider-base](https://app.codecov.io/gh/canonical/checkbox/pull/1455/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | `24.04% <ø> (ø)` | | | [provider-certification-client](https://app.codecov.io/gh/canonical/checkbox/pull/1455/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | `57.14% <ø> (ø)` | | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

eugene-yujinwu commented 1 month ago

Thank you @hanhsuan. @pieqq Could you help review and merge this PR, many thanks!

eugene-yujinwu commented 1 month ago

Based on the latest updates from the client and PM, this test will only serve as a cert block project for desktop. For other projects, it will only be an optional feature. Since this is a client requirement rather than a Canonical requirement, we will remove the cert block tag.

eugene-yujinwu commented 1 month ago

Thank you for your carefully review. @pieqq

eugene-yujinwu commented 1 month ago

The list-bootstrapped and expand results:

u@Drift2-P-2:~$ checkbox-cli list-bootstrapped com.canonical.certification::ethernet-wake-on-lan-cert-manual Skipped file: /usr/share/checkbox-provider-base/units/camera/README.rst Skipped file: /usr/share/checkbox-provider-base/units/snapd/README.md Skipped file: /usr/share/checkbox-provider-base/units/stress/suspend_cycles_reboot.md com.canonical.plainbox::manifest com.canonical.certification::ethernet/wol_S5_enp0s31f6 com.canonical.certification::ethernet/wol_S3_enp0s31f6

u@Drift2-P-2:~$ checkbox-cli expand com.canonical.certification::ethernet-wake-on-lan-cert-manual Skipped file: /usr/share/checkbox-provider-base/units/camera/README.rst Skipped file: /usr/share/checkbox-provider-base/units/snapd/README.md Skipped file: /usr/share/checkbox-provider-base/units/stress/suspend_cycles_reboot.md Template 'com.canonical.certification::ethernet/wol_S3_interface' Template 'com.canonical.certification::ethernet/wol_S5_interface'