Outside of testing contexts, behaving the same as a normal loop and not handling failures should be the default behavior.
Having parameterize continue looping means pressing forward for any failure, even Errors which are not meant to be caught normally. Additionally, having the default onComplete behavior be to throw a ParameterizeFailedError is problematic for the same reason. With that and the additional overhead of recording failures and parameter arguments, the current default behavior in v0.2 isn't suitable for normal use in code.
Changing the default behavior to throw immediately from onFailure means the default parameterize acts identically to a nested for loop, with Errors not being caught, no additional failure recording overhead, and no errors thrown (since throwing from onFailure terminates parameterize without onComplete being executed).
For testing, this is fine since there are two main styles of usage I see. Annotation-based, like kotlin.test/JUnit, and DSL based like Kotest.
For the annotation based frameworks, the suggestion is already to create a shared configuration to hook into the BeforeTest/AfterTest lifecycle. In these cases, it's easy enough to also add the explicit onFailure configured to record failures without breaking early. This also means the old "sane default" of recording only the first 10 failures can be offloaded to the developers, letting them choose what's best.
And for DSL frameworks, having tests declared withinparameterize means the loop-like default is also preferable here too. Failures are not meant to occurin the test DSL itself, so letting the error go uncaught like a loop is preferable here too.
Outside of testing contexts, behaving the same as a normal loop and not handling failures should be the default behavior.
Having
parameterizecontinue looping means pressing forward for any failure, evenErrors which are not meant to be caught normally. Additionally, having the defaultonCompletebehavior be to throw aParameterizeFailedErroris problematic for the same reason. With that and the additional overhead of recording failures and parameter arguments, the current default behavior in v0.2 isn't suitable for normal use in code.Changing the default behavior to throw immediately from
onFailuremeans the defaultparameterizeacts identically to a nested for loop, withErrors not being caught, no additional failure recording overhead, and no errors thrown (since throwing from onFailure terminatesparameterizewithoutonCompletebeing executed).For testing, this is fine since there are two main styles of usage I see. Annotation-based, like kotlin.test/JUnit, and DSL based like Kotest.
BeforeTest/AfterTestlifecycle. In these cases, it's easy enough to also add the explicitonFailureconfigured to record failures without breaking early. This also means the old "sane default" of recording only the first 10 failures can be offloaded to the developers, letting them choose what's best.parameterizemeans the loop-like default is also preferable here too. Failures are not meant to occurin the test DSL itself, so letting the error go uncaught like a loop is preferable here too.