search

canonical / checkbox

Checkbox is a testing framework used to validate device compatibility with Ubuntu Linux. It’s the testing tool developed for the purposes of the Ubuntu Certification program.
https://checkbox.readthedocs.io
GNU General Public License v3.0
33 stars 49 forks source link

Restore all jobs in the job state map (bugfix) #1605

Closed Hook25 closed 1 day ago

Hook25 commented 3 days ago

Description

This fixes the bug with the session that when a test is de-selected, any resume of the session will make the submission.json generation fail. This was because the job_state_map is not entirely re-loaded after the resume, but only the tests that were selected are correctly put in the map.

Additionally, the validation of tests would not cover rejected tests (which is dangerous, as a test may be deselected for a reason, and that reason may not hold anymore).

Finally there was a bug in the session resume where tests were only incidentally not re-run as the rejected job list was not re-evaluated after resume.

Minor: this fixes a bug with ExpectNot, as it wouldn't work if the checkbox controller didnt finish running, now it does

Resolved issues

Fixes: CHECKBOX-1627 Fixes: https://github.com/canonical/checkbox/issues/1557

Documentation

N/A

Tests

This adds a new metabox scenario for the regression

codecov[bot] commented 2 days ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 48.04%. Comparing base (542c748) to head (f9c2870). Report is 9 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #1605 +/- ## ======================================= Coverage 48.03% 48.04% ======================================= Files 371 371 Lines 39852 39855 +3 Branches 6734 6736 +2 ======================================= + Hits 19142 19147 +5 + Misses 19993 19992 -1 + Partials 717 716 -1 ``` | [Flag](https://app.codecov.io/gh/canonical/checkbox/pull/1605/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | Coverage Δ | | |---|---|---| | [checkbox-ng](https://app.codecov.io/gh/canonical/checkbox/pull/1605/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical) | `68.67% <100.00%> (+0.02%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=canonical#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

🚨 Try these New Features: