Fixes flacky tests

[akudiyar]

Let's do this for the tests in other modules too, if it's not required by the test case.

Although in future we'll have to test the full configuration too, especially for cluster metadata related features.