Add support for cleanup steps

[vjt]

This implements feature request #143.

Missing:

• handling the pause event while in cleanup steps - DONE
• handling errors in cleanup steps - DONE by ignoring them
• unit tests - DONE
• integration tests - DONE

[mimir-d]

youve a broken test; what should i understand from the "missing" section above? option 1. you'd like this PR merged and then you add others separately (in which case, i need to check if this one is functionally complete for what it does). or option 2. this a WIP PR and you'll add more commits to it

[vjt]

youve a broken test;

working on fixing that

what should i understand from the "missing" section above? option 1. you'd like this PR merged and then you add others separately (in which case, i need to check if this one is functionally complete for what it does). or option 2. this a WIP PR and you'll add more commits to it

sorry, I wrote that in a hurry. this is a WIP PR and I'll add more commits to it. I'd like your review on what it has been done so far, whether it is reasonable or not, and your insights on the missing items :-)

Thank you!

[codecov-commenter]

Codecov Report

Patch coverage: 81.06% and project coverage change: +0.13 :tada:

Comparison is base (4d6d981) 63.17% compared to head (165eac1) 63.30%.

:exclamation: Current head 165eac1 differs from pull request most recent head 23654f4. Consider uploading reports for the commit 23654f4 to get more accurate results

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## develop #163 +/- ## =========================================== + Coverage 63.17% 63.30% +0.13% =========================================== Files 165 165 Lines 10473 10587 +114 =========================================== + Hits 6616 6702 +86 - Misses 3120 3140 +20 - Partials 737 745 +8 ``` | Flag | Coverage Δ | | |---|---|---| | e2e | `48.51% <59.48%> (+0.05%)` | :arrow_up: | | integration | `54.49% <76.51%> (+0.15%)` | :arrow_up: | | unittests | `49.54% <58.46%> (+0.02%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#carryforward-flags-in-the-pull-request-comment) to find out more. | [Impacted Files](https://app.codecov.io/gh/linuxboot/contest/pull/163?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | Coverage Δ | | |---|---|---| | [pkg/job/events.go](https://app.codecov.io/gh/linuxboot/contest/pull/163?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-cGtnL2pvYi9ldmVudHMuZ28=) | `86.66% <ø> (ø)` | | | [pkg/test/step.go](https://app.codecov.io/gh/linuxboot/contest/pull/163?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-cGtnL3Rlc3Qvc3RlcC5nbw==) | `77.77% <ø> (ø)` | | | [pkg/test/test.go](https://app.codecov.io/gh/linuxboot/contest/pull/163?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-cGtnL3Rlc3QvdGVzdC5nbw==) | `33.33% <ø> (ø)` | | | [plugins/testfetchers/uri/uri.go](https://app.codecov.io/gh/linuxboot/contest/pull/163?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-cGx1Z2lucy90ZXN0ZmV0Y2hlcnMvdXJpL3VyaS5nbw==) | `3.57% <0.00%> (ø)` | | | [tests/integ/jobmanager/jobdescriptors.go](https://app.codecov.io/gh/linuxboot/contest/pull/163?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-dGVzdHMvaW50ZWcvam9ibWFuYWdlci9qb2JkZXNjcmlwdG9ycy5nbw==) | `71.42% <ø> (ø)` | | | [plugins/testfetchers/literal/literal.go](https://app.codecov.io/gh/linuxboot/contest/pull/163?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-cGx1Z2lucy90ZXN0ZmV0Y2hlcnMvbGl0ZXJhbC9saXRlcmFsLmdv) | `62.50% <50.00%> (ø)` | | | [pkg/runner/test\_runner.go](https://app.codecov.io/gh/linuxboot/contest/pull/163?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-cGtnL3J1bm5lci90ZXN0X3J1bm5lci5nbw==) | `85.57% <61.90%> (-1.55%)` | :arrow_down: | | [pkg/jobmanager/job.go](https://app.codecov.io/gh/linuxboot/contest/pull/163?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-cGtnL2pvYm1hbmFnZXIvam9iLmdv) | `63.82% <72.50%> (+3.33%)` | :arrow_up: | | [pkg/pluginregistry/bundles.go](https://app.codecov.io/gh/linuxboot/contest/pull/163?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-cGtnL3BsdWdpbnJlZ2lzdHJ5L2J1bmRsZXMuZ28=) | `63.85% <75.00%> (+0.99%)` | :arrow_up: | | [pkg/runner/job\_runner.go](https://app.codecov.io/gh/linuxboot/contest/pull/163?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None#diff-cGtnL3J1bm5lci9qb2JfcnVubmVyLmdv) | `79.84% <75.90%> (-1.00%)` | :arrow_down: | | ... and [10 more](https://app.codecov.io/gh/linuxboot/contest/pull/163?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None) | | ... and [1 file with indirect coverage changes](https://app.codecov.io/gh/linuxboot/contest/pull/163/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=None)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[mimir-d]

please also change this to merge into develop instead of main

[vjt]

Tests finally pass! 0f7af7c was particularly painful, because I was thinking to make a change affecting only test code, but in reality the code that fetches from storages is the same across tests and prod. Anyway, the change looks reasonable to me.

Cheers

[mimir-d]

thanks for working on this