search

openzfs / zfs

OpenZFS on Linux and FreeBSD
https://openzfs.github.io/openzfs-docs
Other
10.69k stars 1.76k forks source link

Tests suggestion: meta level to exercise permutations of tunables #9652

Open gdevenyi opened 5 years ago

gdevenyi commented 5 years ago

I noticed at https://github.com/zfsonlinux/zfs/pull/9648#issuecomment-559634577

That this bug was only exposed when a tunable was flipped.

That got me thinking that perhaps there are bug lurking in all the possible combinations of tunables, or as in the case of the referenced bug, where tunables are changed during runtime (where allowable). This would likely be an immense amount of tooling, but perhaps the test suite should have the ability to exercise the tunables before/during runs to shake out any bugs. I realize this is likely to be an exponential increase in the number of tests but it might be worth doing once in a while to see what shakes out.

PrivatePuffin commented 5 years ago

How this is currently done in the compression/rsend suite, that multiple loops are ran, each with their own set of random options but a limited number of loops is ran.

Lets say one runs 4 loops, it's actually 4*num_distro's Which is a reasonable amount of tests.

Add to this the number of tests run a day and you would basically end up with your exponential number of permutations, without directly having to run them all each test :)

behlendorf commented 4 years ago

We do a little bit of this kind of testing today with ztest which modifies some default values. If ztest were rock solid today I could imagine it randomly toggling some of these settings to stress test some less used code paths.