AArnott / Xunit.Combinatorial

Adds combinatorial and pairwise testing capability to Xunit tests
Other
180 stars 16 forks source link

CombinatorialValues(null) breaks generation of combinations #20

Closed ReinventorOfWheels closed 4 years ago

ReinventorOfWheels commented 4 years ago

Hello I'm very happy with this package, but I found a bug. This code does not produce a combination of 3 tests as expected:

[Theory]
[CombinatorialData]
public void CombinatorialBug(
   [CombinatorialValues(1, 2, 3)] int paramA,
   [CombinatorialValues(null)] string paramB)
{ }

while this works correctly

[Theory]
[CombinatorialData]
public void CombinatorialBug(
   [CombinatorialValues(1, 2, 3)] int paramA,
   [CombinatorialValues("foo")] string paramB)
{ }

and this too (6 tests)

[Theory]
[CombinatorialData]
public void CombinatorialBug(
  [CombinatorialValues(1, 2, 3)] int paramA,
  [CombinatorialValues(null,"foo")] string paramB  )
{ }
AArnott commented 4 years ago

This isn't a bug in the library. It's a nuance of how C# does params attributes. If you pass null in for a params attribute, instead of turning that into new object[] { null }, C# just passes in null itself. The attribute rejects a null array today because it wants an array of objects.

So this works fine:

[Theory]
[CombinatorialData]
public void CombinatorialBug(
   [CombinatorialValues(1, 2, 3)] int paramA,
   [CombinatorialValues(new object[] { null })] string paramB)
{ }

That said, we can probably re-interpret a null array as an array with a single null element to make this easier.