canonical / checkbox

Checkbox is a testing framework used to validate device compatibility with Ubuntu Linux. It’s the testing tool developed for the purposes of the Ubuntu Certification program.
https://checkbox.readthedocs.io
GNU General Public License v3.0
34 stars 50 forks source link

[checkbox-ce-oem] Revise the test plan structure of strict confinement test (Bugfix) #1419

Closed rickwu666666 closed 3 months ago

rickwu666666 commented 3 months ago

Description

Revise the test plan structure of strict confinement test to make it more clear.

Resolved issues

Documentation

Tests

Side load result of list-bootstrapped

iotuc@atlashost:~$ sudo checkbox-shiner.checkbox-cli list-bootstrapped com.canonical.contrib::strict-confinement-full
Using sideloaded provider: checkbox-provider-ce-oem, version 0.1 from /var/tmp/checkbox-providers/checkbox-provider-ce-oem
Using sideloaded provider: checkbox-provider-shiner, version 0.1 from /var/tmp/checkbox-providers/checkbox-provider-shiner
Skipped file: /snap/checkbox-shiner/335/providers/plainbox-provider-checkbox/units/stress/suspend_cycles_reboot.md
Using sideloaded provider: checkbox-provider-ce-oem, version 0.1 from /var/tmp/checkbox-providers/checkbox-provider-ce-oem
Using sideloaded provider: checkbox-provider-shiner, version 0.1 from /var/tmp/checkbox-providers/checkbox-provider-shiner
com.canonical.certification::init-boot-loop-data
com.canonical.certification::snap
com.canonical.certification::lsb
com.canonical.plainbox::manifest
com.canonical.certification::connections
com.canonical.contrib::strict-confinement/mediacard/sdhc-insert
com.canonical.contrib::strict-confinement/mediacard/sdhc-storage
com.canonical.contrib::strict-confinement/mediacard/sdhc-remove
com.canonical.contrib::strict-confinement/timedatectl-timezone
com.canonical.contrib::strict-confinement/timedatectl-ntp
com.canonical.contrib::strict-confinement/temperature_thermal_zone0_cpu-thermal
com.canonical.certification::sleep
com.canonical.certification::rtc
com.canonical.contrib::strict-confinement/temperature_thermal_zone1_soc-thermal
com.canonical.certification::suspend/suspend_advanced_auto
com.canonical.contrib::after-suspend-strict-confinement/temperature_thermal_zone0_cpu-thermal
com.canonical.contrib::after-suspend-strict-confinement/temperature_thermal_zone1_soc-thermal
com.canonical.contrib::strict-confinement/dbus-warm-boot
com.canonical.contrib::strict-confinement/dbus-warm-boot-test
com.canonical.contrib::strict-confinement/dbus-cold-boot
com.canonical.contrib::strict-confinement/dbus-cold-boot-test
com.canonical.contrib::after-suspend-strict-confinement/mediacard/sdhc-insert
com.canonical.contrib::after-suspend-strict-confinement/mediacard/sdhc-storage
com.canonical.contrib::after-suspend-strict-confinement/mediacard/sdhc-remove
com.canonical.contrib::after-suspend-strict-confinement/timedatectl-timezone
com.canonical.contrib::after-suspend-strict-confinement/timedatectl-ntp

Sideload test result: https://certification.canonical.com/hardware/202403-33886/submission/386826/

stanley31huang commented 3 months ago

can we have manual and automated pattern in the sub test plans? e.g. strict-confinement-mediacard-manual

rickwu666666 commented 3 months ago

can we have manual and automated pattern in the sub test plans? e.g. strict-confinement-mediacard-manual

@stanley31huang I add the postfix to job id and pushed the fix.