xunit / visualstudio.xunit

VSTest runner for xUnit.net (for Visual Studio Test Explorer and dotnet test)
https://xunit.net/
Other
144 stars 81 forks source link

Reporters broken in 2.5.1 release #384

Closed csMACnz closed 11 months ago

csMACnz commented 11 months ago

I'm still investigating the actual root cause or workarounds.

The symptom I am having is that the teamcity reporter is no longer working in xunit.runner.visualstudio 2.5.1. This was working in xunit.runner.visualstudio 2.5.0.

I do get this error message when using -- xUnit.ReporterSwitch=teamcity

[xUnit.net 00:00:00.00] xUnit.net VSTest Adapter v2.5.1.1+5fe7eebc65 (64-bit .NET 7.0.11)
[xUnit.net 00:00:00.02] Could not find requested reporter 'teamcity'

I've narrowed down the difference to the fact that the dlls are no longer included in the build folder of the nuget package. Though still investigating if that is the true cause or not.

image

Which is these commits: https://github.com/xunit/visualstudio.xunit/commit/0d583f282abe40548466205e7328cc9e39cd3d94 https://github.com/xunit/visualstudio.xunit/commit/96555ae16f077124a853ffad1a643d990a331ce8

Installing xunit.runner.reporters into the project has not fixed the issue but testing continues.

Runtime: .Net 7.0

csMACnz commented 11 months ago

After some Investigation, the best I can tell is that xunit.runner.visualstudio (xunit.runner.visualstudio.dotnetcore.testadapter.dll) which lives in the build folder is using that build folder for the reflection and so when xunit.runner.reporters.netcoreapp10.dll is not present in that folder the reporters from inside it are not discovered.

I haven't looked and checked out the discovery code to see if or how it could look inside the test project bin folder for those missing dlls or not... (or if this is the true issue) There is certainly some churn in this space recently though... so maybe it wasn't 100% completed before these dlls were removed?

bradwilson commented 11 months ago

@csMACnz This probably came from #383.

bradwilson commented 11 months ago

Fixed in 2.5.2-pre.3

csMACnz commented 11 months ago

I can confirm that the changes in 2.5.2-pre.3 have fixed this issue 🎉

Thanks for the prompt resolution, looking forward to 2.5.2 to drop 🙇‍♂️

bradwilson commented 11 months ago

I'll wait a few days to see if any other issues surface, and then just cut it over.

Thanks!

nvborisenko commented 11 months ago

It affected 3rd-party reporter https://github.com/reportportal/agent-net-xunit/issues/27. It worked good with 2.5.0.

File name of that reporter is ReportPortal.XUnit.reporters.dll which is located in output bin folder. Tests target net7.0.

I also tried 2.5.2-pre.3, still doesn't work.

bradwilson commented 11 months ago

@nvborisenko Please open a new issue with a repro project. Thanks!

Aleh-Yanushkevich commented 11 months ago

@bradwilson issue #385 with a link to repro project is created.