search

SUSE / rookcheck

Apache License 2.0
4 stars 11 forks source link

Test both openSUSE 15.1 and 15.2 #171

Closed brunoleon closed 4 years ago

brunoleon commented 4 years ago

Previous commit replaced on for the other. It is better to test both.

jhesketh commented 4 years ago

Not sure I agree sorry. The test matrix of possible OS's, kubernetes distros, and rook distros is far too large to cover everything.

We should select carefully which combinations we want to test to 1) make sure we uncover the most bugs as possible, and 2) make sure our product is as well covered as possible. This needs to be balanced with resource usage.

Testing on openSUSE is a way to test upstream rook changes. I'm not sure we gain enough by testing multiple versions of openSUSE, especially since they would both be deployed with the same upstream kubernetes.

For the moment I feel that we have a good enough test matrix for the types of tests we have. We need to see more tests that exercise host-level scenarios (for example, what happens during a kernel panic), to then decide if the way openSUSE 15.1 reacts to it vs. 15.2 if worth testing.

So I would say we want this on the back-burner until we see more tests (and I'm not yet convinced we'll see the types of tests that will get a large gain from this).

brunoleon commented 4 years ago

Yep that is fine for me.

I agree this will be consuming resources.

May be then testing the main version of each major release (15.2, 16.1 when it comes etc...)