fedimint / fedimint

Federated E-Cash Mint
https://fedimint.org/
MIT License
536 stars 209 forks source link

CI: daily workflow testing more versions in back-compat #5180

Open bradleystachurski opened 2 weeks ago

bradleystachurski commented 2 weeks ago

Now that we're using self-hosted CI runners, we should create a daily workflow that runs backwards-compatibility tests against more versions.

CI for PRs only test compatibility vs the oldest supported version (v0.2.1 as-of creating this issue). We originally decided to run against fewer versions to improve developer experience for PRs, but this makes it possible for backwards-incompatible changes to go undetected (see: https://github.com/fedimint/fedimint/pull/5171).

We should also consider other tests that don't fit within PR CI workflows but would be useful to run in daily workflows, e.g. upgrade tests, but that's outside the scope of this issue.

Other considerations:

elsirion commented 2 weeks ago

Yeah, I think we could make the hourly check more extensive but reduce frequency, possibly to daily or a few times around low demand hours.