The setting of the Finished event here does not account for exceptions in the method call above. This was causing the discovery with VS Test Platform to never complete because xunit does not return back a discovery complete message.
In our specific case the problem was that a stale binary referenced an older xunit.core assembly (2.4.1-pre.build.4059, which interestingly had a higher assembly version than the GA release, 2.4.1 (which is why binding redirection entries didn't help as well)). This was causing an assembly load failure at runtime with the following stack
System.IO.FileLoadException
HResult=0x80131040
Message=Could not load file or assembly 'xunit.core, Version=2.4.1.4059, Culture=neutral, PublicKeyToken=8d05b1bb7a6fdb6c' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
mscorlib.dll!System.Reflection.RuntimeAssembly.GetType(string name, bool throwOnError, bool ignoreCase) Unknown
mscorlib.dll!System.Reflection.Assembly.GetType(string name) Unknown
xunit.execution.desktop.dll!Xunit.Sdk.SerializationHelper.GetType(string assemblyName, string typeName) Line 189 C#
> xunit.execution.desktop.dll!Xunit.Sdk.ExtensibilityPointFactory.GetTraitDiscoverer(Xunit.Abstractions.IMessageSink diagnosticMessageSink, Xunit.Abstractions.IAttributeInfo traitDiscovererAttribute) Line 199 C#
xunit.execution.desktop.dll!Xunit.Sdk.XunitTestCase.Initialize() Line 134 C#
xunit.execution.desktop.dll!Xunit.Sdk.TestMethodTestCase.UniqueID.get() Line 184 C#
xunit.execution.desktop.dll!Xunit.Sdk.TestCaseDescriptorFactory.TestCaseDescriptorFactory(object discovererObject, object testCasesObject, object callbackObject) Line 33 C#
[Native to Managed Transition]
[Managed to Native Transition]
mscorlib.dll!System.RuntimeType.CreateInstanceImpl(System.Reflection.BindingFlags bindingAttr, System.Reflection.Binder binder, object[] args, System.Globalization.CultureInfo culture, object[] activationAttributes, ref System.Threading.StackCrawlMark stackMark) Unknown
mscorlib.dll!System.Activator.CreateInstance(System.Type type, System.Reflection.BindingFlags bindingAttr, System.Reflection.Binder binder, object[] args, System.Globalization.CultureInfo culture, object[] activationAttributes) Unknown
mscorlib.dll!System.Activator.CreateInstance(string assemblyString, string typeName, bool ignoreCase, System.Reflection.BindingFlags bindingAttr, System.Reflection.Binder binder, object[] args, System.Globalization.CultureInfo culture, object[] activationAttributes, System.Security.Policy.Evidence securityInfo, ref System.Threading.StackCrawlMark stackMark) Unknown
mscorlib.dll!System.Activator.CreateInstance(string assemblyName, string typeName, bool ignoreCase, System.Reflection.BindingFlags bindingAttr, System.Reflection.Binder binder, object[] args, System.Globalization.CultureInfo culture, object[] activationAttributes, System.Security.Policy.Evidence securityInfo) Unknown
mscorlib.dll!System.AppDomain.CreateInstance(string assemblyName, string typeName, bool ignoreCase, System.Reflection.BindingFlags bindingAttr, System.Reflection.Binder binder, object[] args, System.Globalization.CultureInfo culture, object[] activationAttributes, System.Security.Policy.Evidence securityAttributes) Unknown
mscorlib.dll!System.AppDomain.CreateInstanceAndUnwrap(string assemblyName, string typeName, bool ignoreCase, System.Reflection.BindingFlags bindingAttr, System.Reflection.Binder binder, object[] args, System.Globalization.CultureInfo culture, object[] activationAttributes, System.Security.Policy.Evidence securityAttributes) Unknown
[AppDomain (testhost.net472.x86.exe, #1) -> AppDomain (Microsoft.CodeAnalysis.CSharp.CommandLine.UnitTests, #87)]
xunit.runner.utility.net452.dll!Xunit.AppDomainManager_AppDomain.CreateObject<object>(System.Reflection.AssemblyName assemblyName, string typeName, object[] args) Line 78 C#
xunit.runner.utility.net452.dll!Xunit.Xunit2Discoverer.GetTestCaseDescriptors(System.Collections.Generic.List<Xunit.Abstractions.ITestCase> testCases, bool includeSerialization) Line 202 C#
xunit.runner.utility.net452.dll!Xunit.XunitFrontController.GetTestCaseDescriptors(System.Collections.Generic.List<Xunit.Abstractions.ITestCase> testCases, bool includeSerialization) Line 184 C#
xunit.runner.visualstudio.testadapter.dll!Xunit.Runner.VisualStudio.VsDiscoverySink.SendExistingTestCases() Line 204 C#
xunit.runner.visualstudio.testadapter.dll!Xunit.Runner.VisualStudio.VsDiscoverySink.HandleTestCaseDiscoveryMessage(Xunit.MessageHandlerArgs<Xunit.Abstractions.ITestCaseDiscoveryMessage> args) Line 184 C#
xunit.runner.utility.net452.dll!MessageSinkMessageExtensions.Dispatch<Xunit.Abstractions.ITestCaseDiscoveryMessage>(Xunit.Abstractions.IMessageSinkMessage message, System.Collections.Generic.HashSet<string> messageTypes, Xunit.MessageHandler<Xunit.Abstractions.ITestCaseDiscoveryMessage> callback) Line 39 C#
xunit.runner.utility.net452.dll!Xunit.DiscoveryEventSink.OnMessageWithTypes(Xunit.Abstractions.IMessageSinkMessage message, System.Collections.Generic.HashSet<string> typeNames) Line 27 C#
xunit.runner.visualstudio.testadapter.dll!Xunit.Runner.VisualStudio.VsDiscoverySink.Xunit.IMessageSinkWithTypes.OnMessageWithTypes(Xunit.Abstractions.IMessageSinkMessage message, System.Collections.Generic.HashSet<string> messageTypes) Line 197 C#
xunit.runner.utility.net452.dll!Xunit.MessageSinkAdapter.OnMessageWithTypes(Xunit.Abstractions.IMessageSinkMessage message, System.Collections.Generic.HashSet<string> messageTypes) Line 50 C#
Although the stack above points to HandleTestCaseDiscoveryMessage the same happens when HandleDiscoveryCompleteMessage is triggered as well.
Logging this issue to help make the discovery process a little more reliable.
The setting of the Finished event here does not account for exceptions in the method call above. This was causing the discovery with VS Test Platform to never complete because xunit does not return back a discovery complete message. In our specific case the problem was that a stale binary referenced an older xunit.core assembly (2.4.1-pre.build.4059, which interestingly had a higher assembly version than the GA release, 2.4.1 (which is why binding redirection entries didn't help as well)). This was causing an assembly load failure at runtime with the following stack
Although the stack above points to
HandleTestCaseDiscoveryMessage
the same happens whenHandleDiscoveryCompleteMessage
is triggered as well.Logging this issue to help make the discovery process a little more reliable.