xunit / visualstudio.xunit

VSTest runner for xUnit.net (for Visual Studio Test Explorer and dotnet test)
https://xunit.net/
Other
144 stars 81 forks source link

Suggest allows variable arguments to be empty #371

Closed xp44mm closed 1 year ago

xp44mm commented 1 year ago

I use F# code to demonstrate the problem:

    [<Theory>]
    [<InlineData()>] // **note**
    [<InlineData(1)>]
    [<InlineData(1,2)>]
    member _.``variable arguments`` ([<System.ParamArray>]sq:int[]) =
        for i in sq do
            output.WriteLine($"{i}")

expected:

    [<InlineData()>]  => sq = [||]

but:

System.InvalidOperationException : The test method expected 1 parameter value, but 0 parameter values were provided.
ghost commented 1 year ago

Is there a way to reproduce this using C#? I actually tried it on my end and I couldn't find a way to reproduce it.

MJMcNazir commented 1 year ago
[<Theory>]
    [<InlineData()>] // **note**
    [<InlineData(1)>]
    [<InlineData(1,2)>]
    member _.``variable arguments`` ([<System.ParamArray>]sq:int[]) =
        for i in sq do
            output.WriteLine($"{i}")

Hi @sdelarosbil, Here is the C# equivalent of the code:

using Xunit;
public class MyTestClass
{
 [Theory]
 [InlineData] // **note**
 [InlineData(1)]
 [InlineData(1,2)]
 public void VariableArgumentsTest(params int[] sq)
 {
     foreach (int i in sq)
      {
         System.Console.WriteLine($"{i}");
      }
 }
}

To run it go to "Test>Run All Tests" or press "Ctrl+R, A" Then open the "Test Explorer" window from "Test>Test Explorer" or press "Ctrl+E, T"

xp44mm commented 1 year ago

output is an input argument of the test class ctor, it is different with System.Console.:

type Test(output:ITestOutputHelper) =
nih0n commented 1 year ago

Hi @xp44mm, could you provide a repository to reproduce the error ? I tried it on my side and it works flawlessly.

bradwilson commented 1 year ago

@nih0n I'm able to reproduce it, but only when using Test Explorer in Visual Studio. dotnet test does not have any problems with it, which implies that problem is a serialization problem.

bradwilson commented 1 year ago

Also, it only seems to affect .NET Core projects. .NET Framework projects don't seem to have the same problem.

image

bradwilson commented 1 year ago

I added a test to see if I could flush this out inside the core framework, but everything seems to be working as expected:

https://github.com/xunit/xunit/blob/b0b8e0d691f68c2ef81edd87a4ce0987b6cf3827/test/test.xunit.execution/Sdk/Frameworks/TheoryDiscovererTests.cs#L233-L276

At this point my suspicion is the bug lives in the VSTest adapter, so I'm transferring this issue over to that repo.

xp44mm commented 1 year ago

@bradwilson Your job is irreplaceable by AI!

bradwilson commented 1 year ago

I'm still not 100% sure why we get where we're getting, but I was able to put a little escape hatch into the system to say "if we end up here, just pretend we got valid data and move on". I'm not 100% pleased with this, but until/unless I can actually figure out why the values from the VSTest adapter come back incorrect, I'm basically stuck with this semi-hack.