search

canonical / checkbox

Checkbox is a testing framework used to validate device compatibility with Ubuntu Linux. It’s the testing tool developed for the purposes of the Ubuntu Certification program.
https://checkbox.readthedocs.io
GNU General Public License v3.0
34 stars 50 forks source link

snap refresh/revert script may fail, but the job is marked as passed regardless #1615

Open pieqq opened 3 days ago

pieqq commented 3 days ago

Bug Description

Problem found while testing a device in the lab (Jenkins sample runs: run 1, run 2).

The job snapd/snap-refresh-kernel-.*-to-base-rev is run.

However, it takes so long that it times out... but somehow it is marked as passed. (this is because the job reboots regardless of the test outcome. It's the issue that needs to be fixed)

Because it was marked as passed, the job snapd/snap-revert-kernel-.*-from-base-rev is erroneously executed, and it fails because it cannot find the revision to revert into (because the device never reached that state).

All the snap refresh/revert jobs that launch the snap_update_test.py script and then reboot the device should first check the return code of the script and bail out (exit 1) if the script fails. These jobs are impacted:

Cert-blocker Test Case

To Reproduce

Launch the snap refresh/revert jobs on a device slow enough to fail the test.

Environment

Relevant log output

No response

Additional context

No response

syncronize-issues-to-jira[bot] commented 3 days ago

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/CHECKBOX-1671.

This message was autogenerated