AArnott / Xunit.Combinatorial

Adds combinatorial and pairwise testing capability to Xunit tests
Other
180 stars 16 forks source link

Add an attribute to mix traditional and combinatorial data #93

Open asimon-od opened 11 months ago

asimon-od commented 11 months ago

Essentially, I'm looking to be able to hand write values for some test parameters while combinatorially populating others. This would allow more control over e.g. int values than the existing range options or the veto option proposed in #3.

An (albeit useless) toy example:

/// Assumption: a > b
[Theory]
[InlineData(10, 0, true)]
[InlineData(10, 0, false)]
[InlineData(5, 2, true)]
[InlineData(5, 2, false)]
public void Example(int a, int b, bool c)
{
  // body
}

In this example, the values for a and b are hand-picked, but should be tested for both values of c. I assume it may not be possible to build this functionality on top of the existing InlineData attribute, but supposing a new keyword-style equivalent that replaced any omitted parameters with their combinatorial definitions, it could look like this:

/// Assumption: a > b
[Theory, CombinatorialData]
[PartialInlineData(10, 0)]
[PartialInlineData(10, 0)]
[PartialInlineData(5, 2)]
[PartialInlineData(5, 2)]
public void Example(int a, int b, [CombinatorialValues(true, false)] bool c)
{
  // body
}

and could also support cases where we don't want the combinatorics a la #3

[PartialInlineData(6, 2, false)]
// No need to veto [PartialInlineData(6, 2, true)] if you don't want it
AArnott commented 9 months ago

It sounds interesting. The combinatorial algorithm is very complex, and I just copied it from NUnit. I'm not sure how easily adaptable it would be to this idea. I think your proposed API would need a little adjusting as well. But if you have time and interest to give it a try, let me know and I'll assign the issue to you. Otherwise I may close the issue as I don't foresee ever having the time to deliver on this myself.

silkfire commented 7 months ago

@asimon-od For now, the workaround for your scenario is to use tuples. Works perfectly although I do understand your request for it to be part of the base API of the library.

[Theory, CombinatorialData]
public void Example([CombinatorialMemberData(nameof(GetIntegerData))](int, int) integerData, bool c)
{
  // body
}

private static IEnumerable<(int, int)> GetIntegerData()
{
    yield return (10, 0);
    yield return (5, 2);
}