search

canonical / checkbox

Checkbox is a testing framework used to validate device compatibility with Ubuntu Linux. It’s the testing tool developed for the purposes of the Ubuntu Certification program.
https://checkbox.readthedocs.io
GNU General Public License v3.0
31 stars 47 forks source link

The test cases amount after first cold boot/ warm boot test will be different while using remote test #976

Open hanhsuan opened 9 months ago

hanhsuan commented 9 months ago

Bug Description

While using checkbox remote testing, the test cases amount will be different after first cold boot/warm boot.

warm boot:

---------------------------[ Running job 221 / 230 ]----------------------------
--------------------------------[ Warm reboot ]---------------------------------
ID: com.canonical.certification::power-management/warm-reboot
Category: Power Management tests
--------------------------------------------------------------------------------
Connection lost!
connection closed by peer
Reconnecting -\|/-\|...
Reconnected (took: 34s)
--------------------------------[ Warm reboot ]---------------------------------
ID: com.canonical.certification::power-management/warm-reboot
Category: Power Management tests
--------------------------------------------------------------------------------
Outcome: job passed
---------------------------[ Running job 232 / 240 ]----------------------------
-----------------------[ Post warm reboot service check ]-----------------------
ID: com.canonical.certification::power-management/post-warm-reboot
Category: Power Management tests
--------------------------------------------------------------------------------
Found 0 failed units
--------------------------------------------------------------------------------
Outcome: job passed
---------------------------[ Running job 233 / 240 ]----------------------------

cold boot

---------------------------[ Running job 210 / 219 ]----------------------------
-------------------------[ power-management/poweroff ]--------------------------
ID: com.canonical.certification::power-management/poweroff
Category: Power Management tests
--------------------------------------------------------------------------------
Reconnected (took: 120s)
-------------------------[ power-management/poweroff ]--------------------------
ID: com.canonical.certification::power-management/poweroff
Category: Power Management tests
--------------------------------------------------------------------------------
Outcome: job passed
---------------------------[ Running job 219 / 227 ]----------------------------

To Reproduce

checkbox-cli run com.canonical.certification::power-automated

Environment

Relevant log output

No response

Additional context

3.3.0~dev107 3.3.0~dev19 1.9.0

syncronize-issues-to-jira[bot] commented 9 months ago

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/CHECKBOX-1225.

This message was autogenerated

pieqq commented 7 months ago

I'm wondering if this is something related to bootstrapped jobs that would be included in the initial count, and not afterwards.