canonical / rockcraft

Tool to create OCI Images using the language from Snapcraft and Charmcraft.
GNU General Public License v3.0
37 stars 44 forks source link

create new Spread task for daily compatibility matrix testing #665

Open cjdcordeiro opened 3 months ago

cjdcordeiro commented 3 months ago

What needs to get done

Rockcraft has several loose dependencies, like LXD, multipass and docker (for the docs).

It would be good to have a dedicated spread test, with variants, to run a basic rockcraft pack on different versions of said dependencies.

Ideally, this test should also be run daily to improve our reaction time to changes happening elsewhere.

Why it needs to get done

Ref issues:

tigarmo commented 3 months ago

I worry about the maintenance burden - we already had to disable the tests that use build-base: devel from the regular run because of interim breakage in 24.10, so in the LXD case we'd have another set of tests failing due to changes outside of our immediate control.

cjdcordeiro commented 3 months ago

But that's precisely why I'd like to have this test. To identify such cases, quickly. Imagine we had this test last week...we'd could've promptly acted (either by fixing the compatibility or updating the docs). And yes, to avoid continuous failures, we need to update this job with a list of supported versions that we'd need to maintain as an auxiliary CI script