search

reportportal / agent-net-vstest

Logger extension for VsTest
Apache License 2.0
8 stars 10 forks source link

Logger gets unhanded exception while extricating fullPath from the full test name (fullName) #28

Closed yurii-hunter closed 5 years ago

yurii-hunter commented 5 years ago

https://github.com/reportportal/agent-net-vstest/blob/a6d01fae68f5c7e231fce21782a58a26e5a78fbc/src/ReportPortal.VSTest.TestLogger/ReportPortalLogger.cs#L124

In case user uses xUnit.net the e.Result.TestCase.DisplayName is equal to e.Result.TestCase.FullyQualifiedName. It causes exception on line number 129 because logger tries to get substring with length -1 

var fullPath = fullName.Substring(0, fullName.Length - testName.Length - 1);

it will became

var fullPath = fullName.Substring(0, - 1);

Looks like the fastest solution is to try to get test name from the DisplayName, but i'm not sure about this.

var fullName = e.Result.TestCase.FullyQualifiedName;
var displayName = e.Result.TestCase.DisplayName ?? fullName;
var testName = displayName.Split('.').Last();

var fullPath = fullName.Substring(0, fullName.Length - testName.Length - 1);
nvborisenko commented 5 years ago

@yurii-hunter could you please share a piece of code how your test looks like which causes this issue?

yurii-hunter commented 5 years ago

Microsoft.NET.Test.Sdk Version=16.2.0 xunit Version = 2.4.1 xunit.runner.visualstudio Version = 2.4.1 Project Type - Class Library (.net core)

[Fact]
public void TestMethod()
{
    Assert.True(false);
}

@nvborisenko vstest logger creates test launch bat can not report any test there

nvborisenko commented 5 years ago

Thanks, will take a look in 1 week when get back to PC.

yurii-hunter commented 5 years ago

jfPrCm1icB

A little bit more details: if DisplayName is not specfified - then FullyQualifiedName equals DisplayName if DisplayName is specified [Fact(DisplayName = "Test.Method")] - then DisplayName shows specified display name "Test.Method"

nvborisenko commented 5 years ago

Right, got the same results. This behavior is specific only foe xunit tests. Not sure whether this should be resolved on our side or xunit adapter side.

Some variants:

fq: ExampleUnitTests.XUnitTests.UnitTest3.Test4, dn: ExampleUnitTests.XUnitTests.UnitTest3.Test4(a: "1.postfix")
fq: ExampleUnitTests.XUnitTests.UnitTest3.Test4, dn: ExampleUnitTests.XUnitTests.UnitTest3.Test4(a: "2.postfix")
fq: ExampleUnitTests.XUnitTests.UnitTest3.Test3, dn: Dot.Test3
fq: ExampleUnitTests.XUnitTests.UnitTest3.Test2, dn: OverriddenTest2
fq: ExampleUnitTests.XUnitTests.UnitTest3.Test1, dn: ExampleUnitTests.XUnitTests.UnitTest3.Test1

where fq is fully qualified name, dn - Display Name.

I don't see how to understand what is namespace and name of test based on this information :(.

We don't meet this behavior in mstest and nunit adapters.

yurii-hunter commented 5 years ago

@nvborisenko i've created PR that fixes this problem for xUnit. But I have not tested it with another frameworks. The result looks like this image

nvborisenko commented 5 years ago

This works for xunit, but doesn't work for nunit. You can play with example-net-vstest repository to verify your package.

yurii-hunter commented 5 years ago

@nvborisenko will do. Thanks for a feedback