canonical / checkbox

Checkbox is a testing framework used to validate device compatibility with Ubuntu Linux. It’s the testing tool developed for the purposes of the Ubuntu Certification program.
https://checkbox.readthedocs.io
GNU General Public License v3.0
33 stars 50 forks source link

Remove require for rt-tests (BugFix) #1553

Open LiaoU3 opened 1 month ago

LiaoU3 commented 1 month ago

Description

Since we have package rt-tests in snapcraft and packaging, there is no need to require the package in the tests.

Resolved issues

N/A

Documentation

N/A

Tests

N/A

pieqq commented 3 weeks ago

Why is this removal needed? Having package.name can ensure the test runs only when that package is available (for instance, when running using a virtual environment).

LiaoU3 commented 3 weeks ago

@pieqq It is not working on the snap version of checkbox, since we package rt-tests into the stage-packages, it is not going to be shown in the resource job package, due to the fact that package only list the debian package installed on the local.

===========[ Running job 2 / 3. Estimated time left: 1 day, 0:01:00 ]===========
[ A short-duration cyclictest to validate the system's real-time capabilities. ]
ID: com.canonical.certification::rt-performance-tests/cyclictest_rt
Category: com.canonical.certification::rt-performance-tests
Job cannot be started because:
 - resource expression "package.name == 'rt-tests'" evaluates to false
Outcome: job cannot be started
===========[ Running job 3 / 3. Estimated time left: 1 day, 0:00:00 ]===========
--[ Run cyclictest to test scheduling latency. Default run time is 24 hours. ]--
ID: com.canonical.certification::rt-performance-tests/cyclictest_rt_latency
Category: com.canonical.certification::rt-performance-tests
Job cannot be started because:
 - resource expression "package.name == 'rt-tests'" evaluates to false
Outcome: job cannot be started
(base) vincent@vincent-XPS-9320:~$ snap list checkbox24 
Name        Version      Rev  Tracking       Publisher            Notes
checkbox24  4.1.0-dev25  75   latest/stable  ce-certification-qa  -
pieqq commented 3 weeks ago

Please use snap resource job instead if this is a snap package to be checked

That, or the executable resource.

LiaoU3 commented 3 weeks ago

executable makes more sense