JetBrains / resharper-unity

Unity support for both ReSharper and Rider
Apache License 2.0
1.21k stars 134 forks source link

Running tests via Unity pending/inconclusive #526

Closed igisav closed 6 years ago

igisav commented 6 years ago

After updgrading to Rider 2018.1 and running unit tests without starting Unity on my Mac give me 'inconclusive' result. If I start Unity, then tests stay in pending mode without any response....

Rider EAP Build #RD-181.4379.1196
Unity: 2018.1.0b12
JRE: 1.8.0_152-release-1136-b23 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
macOS 10.12.6

a part of stacktrace:

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.InvalidCastException: Specified cast is not valid.
  at NUnit.Framework.Api.FrameworkController+LoadTestsAction..ctor (NUnit.Framework.Api.FrameworkController controller, System.Object handler) [0x00008] in <d9eafad60e01411f9e6f726dccd715f8>:0 
  at (wrapper managed-to-native) System.Reflection.MonoCMethod.InternalInvoke(System.Reflection.MonoCMethod,object,object[],System.Exception&)
  at System.Reflection.MonoCMethod.InternalInvoke (System.Object obj, System.Object[] parameters) [0x00002] in /Users/builder/data/lanes/5533/mono-mac-sdk/external/bockbuild/builds/mono-x64/mcs/class/corlib/System.Reflection/MonoMethod.cs:661 
van800 commented 6 years ago

@igisav For all Unity versions 5.6 and above currently we don't have a good solution, which would allow running nunit tests in Unity-generater solution without Unity running.

In the past we tried to update nunit-framework.dll from Unity one to the original one, but it caused side-effects https://youtrack.jetbrains.com/issue/RIDER-14825

I hope we finally will find a consistent solution.

van800 commented 6 years ago

What if you have Unity and Rider running and connection indicator shows "Connected" and then run tests? Does it work for you?

igisav commented 6 years ago

@van800 No. If I run test with "Connected", I see pending state of tests, then progression clock on Unity icon, and then tests converted to inconclusive. Some strange Rider exceptions popups (see below) But after some little changes in code, recompiling and switching between Rider and Unity, suddenly tests are running....I could not find a reasonable explanation.

--- EXCEPTION #1/2 [InvalidOperationException]
Message = “An attempt was made to transition a task to a final state when it had already completed.”
ExceptionPath = Root.InnerException
ClassName = System.InvalidOperationException
HResult = COR_E_INVALIDOPERATION=80131509
Source = mscorlib
StackTraceString = “
  at System.Threading.Tasks.TaskCompletionSource`1[TResult].SetResult (TResult result) [0x00013] in <4343323a00204b0b9f5e7ca3dfd716be>:0 
    at JetBrains.ReSharper.Plugins.Unity.Rider.UnitTesting.RunViaUnityEditorStrategy+<>c__DisplayClass11_0.<RunInternal>b__1 (JetBrains.Platform.Unity.EditorPluginModel.RunResult result) [0x00000] in <7784dad26cd94e81b5185f57617e441d>:0 
    at JetBrains.Platform.RdFramework.Util.Signal`1[T].Fire0 (System.Action`1[T][] listeners, T value) [0x0000a] in <97f41334bc0f41e7ba293e79afec26b2>:0
”

--- Outer ---

--- EXCEPTION #2/2 [LoggerException]
Message = “An attempt was made to transition a task to a final state when it had already completed.”
ExceptionPath = Root
ClassName = JetBrains.Util.LoggerException
InnerException = “Exception #1 at Root.InnerException”
HResult = COR_E_APPLICATION=80131600
StackTraceString = “
  at JetBrains.Util.LoggerBase.Log (JetBrains.Util.LoggingLevel level, System.String message, System.Exception ex) [0x00000] in <6045c331b4334e3e8f3f917b0d597942>:0 
    at JetBrains.Util.Logging.Log+SwitchingLog.JetBrains.Util.Logging.ILog.Log (JetBrains.Util.LoggingLevel level, System.String message, System.Exception exception) [0x00000] in <983372af6d8a4f249425ee2c68b3cd00>:0 
    at JetBrains.Util.Logging.LogEx.Error (JetBrains.Util.Logging.ILog this, System.Exception ex, System.String message) [0x00000] in <983372af6d8a4f249425ee2c68b3cd00>:0 
    at JetBrains.Platform.RdFramework.Util.Signal`1[T].Fire0 (System.Action`1[T][] listeners, T value) [0x00000] in <97f41334bc0f41e7ba293e79afec26b2>:0 
    at JetBrains.Platform.RdFramework.Util.Signal`1[T].Fire (T value) [0x00000] in <97f41334bc0f41e7ba293e79afec26b2>:0 
    at JetBrains.Platform.RdFramework.Impl.RdSignal`1+<>c__DisplayClass12_0[T].<.ctor>b__1 (JetBrains.Util.PersistentMap.UnsafeReader stream) [0x00000] in <97f41334bc0f41e7ba293e79afec26b2>:0 
    at JetBrains.Platform.RdFramework.Impl.MessageBroker.Execute (System.Action`1[T] handler, JetBrains.Platform.RdFramework.RdId id, System.Byte[] msg) [0x00000] in <97f41334bc0f41e7ba293e79afec26b2>:0 
    at JetBrains.Platform.RdFramework.Impl.MessageBroker+Subscription+<>c__DisplayClass5_0.<Invoke>b__0 () [0x00000] in <97f41334bc0f41e7ba293e79afec26b2>:0 
van800 commented 6 years ago

It may be related to the way we install plugin. I would speed up improvement https://github.com/JetBrains/resharper-unity/issues/510 and then I guess the problem will disappear.

van800 commented 6 years ago

I am keeping this issue for running simple unit-tests without connected Unity.

van800 commented 6 years ago

duplicated by https://youtrack.jetbrains.com/issue/RIDER-15415

tactile-readonly commented 6 years ago

@igisav I have tried to close both applications, pressing the sync button in unity, switch back and forth many times, recompile both assemblies several times.. i can not make it work. Can you remember if you did something special?

trondtactile commented 6 years ago

I have not been able to make the test runner work. Tried "steps" above, as in restart, recompile, reconnect etc, still not running the test, only "pending". Both applications are running, and rider states its connected to rider.

van800 commented 6 years ago

I reproduced the problem with Unity 5.6.3.

[ERROR]14:52:40.687 
Could not load file or assembly 'UnityEngine.TestRunner, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies.
  at JetBrains.Rider.Unity.Editor.Ge56.UnitTesting.ModelAdviceExtension+<>c.<AdviseUnitTestLaunch>b__0_0 (JetBrains.Platform.Unity.EditorPluginModel.UnitTestLaunch launch) [0x00000] in <filename unknown>:0 
  at JetBrains.Platform.RdFramework.Util.Signal`1[JetBrains.Platform.Unity.EditorPluginModel.UnitTestLaunch].Fire0 (System.Action`1[] listeners, JetBrains.Platform.Unity.EditorPluginModel.UnitTestLaunch value) [0x00000] in <filename unknown>:0 

@tactile-readonly what is your Unity version?

tactile-readonly commented 6 years ago

@van800 we currently use 5.6.3p4.

igisav commented 6 years ago

@tactile-readonly try to do some changes in your code (i.e. unit tests) and run test runner in Unity. But, anyway, I don't know... it's like alchemy and guessing

van800 commented 6 years ago

Fix for Unity 5.6.3 is on the way https://github.com/JetBrains/resharper-unity/pull/531

trondtactile commented 6 years ago

Fantastic, thank you!

van800 commented 6 years ago

Nightly build with fix (Unity 5.6.3 problem) is already available via https://www.jetbrains.com/toolbox/app/ Please try and let us know, how it goes.

trondtactile commented 6 years ago

So, running basic unit tests now works. However, for tests using the [Values()] attribute, it says "Test not run".

Got this exception in Unity:

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> Mono.Cecil.AssemblyResolutionException: Failed to resolve assembly: 'System.Runtime, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' at Mono.Cecil.BaseAssemblyResolver.Resolve (Mono.Cecil.AssemblyNameReference name, Mono.Cecil.ReaderParameters parameters) [0x000f7] in :0 at Mono.Cecil.BaseAssemblyResolver.Resolve (Mono.Cecil.AssemblyNameReference name) [0x00007] in :0 at Mono.Cecil.DefaultAssemblyResolver.Resolve (Mono.Cecil.AssemblyNameReference name) [0x00025] in :0 at Mono.Cecil.MetadataResolver.Resolve (Mono.Cecil.TypeReference type) [0x00040] in :0 at Mono.Cecil.ModuleDefinition.Resolve (Mono.Cecil.TypeReference type) [0x00006] in :0 at Mono.Cecil.TypeReference.Resolve () [0x00006] in :0 at Mono.Cecil.Mixin.CheckedResolve (Mono.Cecil.TypeReference self) [0x00000] in :0 at Mono.Cecil.SignatureReader.ReadCustomAttributeEnum (Mono.Cecil.TypeReference enum_type) [0x00000] in :0 at Mono.Cecil.SignatureReader.ReadCustomAttributeElementValue (Mono.Cecil.TypeReference type) [0x0002f] in :0 at Mono.Cecil.SignatureReader.ReadCustomAttributeElement (Mono.Cecil.TypeReference type) [0x00015] in :0 at Mono.Cecil.SignatureReader.ReadCustomAttributeFixedArgument (Mono.Cecil.TypeReference type) [0x00015] in :0 at Mono.Cecil.SignatureReader.ReadCustomAttributeConstructorArguments (Mono.Cecil.CustomAttribute attribute, Mono.Collections.Generic.Collection1[T] parameters) [0x0002e] in <c6cbe17310154746b5cee213d0ed414a>:0 at Mono.Cecil.MetadataReader.ReadCustomAttributeSignature (Mono.Cecil.CustomAttribute attribute) [0x0003c] in <c6cbe17310154746b5cee213d0ed414a>:0 at Mono.Cecil.CustomAttribute.<Resolve>b__35_0 (Mono.Cecil.CustomAttribute attribute, Mono.Cecil.MetadataReader reader) [0x00000] in <c6cbe17310154746b5cee213d0ed414a>:0 at Mono.Cecil.ModuleDefinition.Read[TItem,TRet] (TItem item, System.Func3[T1,T2,TResult] read) [0x00029] in :0 at Mono.Cecil.CustomAttribute.Resolve () [0x00017] in :0 at Mono.Cecil.CustomAttribute.get_ConstructorArguments () [0x00000] in :0 at Unity.Cecil.Visitor.Visitor.Visit (Mono.Cecil.CustomAttribute customAttribute, Unity.Cecil.Visitor.Context context) [0x0003e] in <79eb5983cd354eedb9ed613dbacce245>:0 at Unity.Cecil.Visitor.Visitor.Visit (Mono.Cecil.MethodDefinition methodDefinition, Unity.Cecil.Visitor.Context context) [0x00037] in <79eb5983cd354eedb9ed613dbacce245>:0 at APIUpdater.Framework.Configuration.ConfigurationCollector.Visit (Mono.Cecil.MethodDefinition methodDefinition, Unity.Cecil.Visitor.Context context) [0x00001] in <79eb5983cd354eedb9ed613dbacce245>:0 at Unity.Cecil.Visitor.Visitor.Visit (Mono.Cecil.TypeDefinition typeDefinition, Unity.Cecil.Visitor.Context context) [0x001b1] in <79eb5983cd354eedb9ed613dbacce245>:0 at APIUpdater.Framework.Configuration.ConfigurationCollector.Visit (Mono.Cecil.TypeDefinition typeDefinition, Unity.Cecil.Visitor.Context context) [0x00001] in <79eb5983cd354eedb9ed613dbacce245>:0 at Unity.Cecil.Visitor.Visitor.Visit (Mono.Cecil.ModuleDefinition moduleDefinition, Unity.Cecil.Visitor.Context context) [0x00024] in <79eb5983cd354eedb9ed613dbacce245>:0 at Unity.Cecil.Visitor.Visitor.Visit (Mono.Cecil.AssemblyDefinition assemblyDefinition, Unity.Cecil.Visitor.Context context) [0x00024] in <79eb5983cd354eedb9ed613dbacce245>:0 at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&) at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00038] in :0 --- End of inner exception stack trace --- at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00053] in :0 at System.Reflection.MethodBase.Invoke (System.Object obj, System.Object[] parameters) [0x00000] in :0 at Unity.Cecil.Visitor.Visitor.Visit[T] (T node, Unity.Cecil.Visitor.Context context) [0x00038] in <79eb5983cd354eedb9ed613dbacce245>:0 at Unity.Cecil.Visitor.Extensions.DoAccept[T] (T definition, Unity.Cecil.Visitor.Visitor visitor) [0x0001e] in <79eb5983cd354eedb9ed613dbacce245>:0 at Unity.Cecil.Visitor.Extensions.Accept (Mono.Cecil.AssemblyDefinition assemblyDefinition, Unity.Cecil.Visitor.Visitor visitor) [0x00001] in <79eb5983cd354eedb9ed613dbacce245>:0 at APIUpdater.Framework.Configuration.ObsoleteAttributeBasedConfigurationFactory.CollectConfigurationFrom (Mono.Cecil.AssemblyDefinition assembly) [0x0000e] in <79eb5983cd354eedb9ed613dbacce245>:0 at APIUpdater.Framework.Configuration.ObsoleteAttributeBasedConfigurationFactory..ctor (System.String pathToAssemblyWithAttr, System.String[] references) [0x00084] in <79eb5983cd354eedb9ed613dbacce245>:0 at APIUpdater.Framework.Util.IEnumerableExtensions.ComputeConfigurationProvider (System.Collections.Generic.IEnumerable1[T] assemblyPaths, APIUpdater.Framework.Configuration.IConfigurationProvider provider) [0x00017] in <79eb5983cd354eedb9ed613dbacce245>:0 at AssemblyUpdater.Application.Program.ConfigurationProviderFrom (System.Collections.Generic.IEnumerable1[T] configurationAssemblies) [0x00020] in <2e2e6e53129e4ba1874d2ad0eec22c59>:0 at AssemblyUpdater.Application.Program.CheckForObsoleteAPIUsage (AssemblyUpdater.Application.CommandLineSpec config) [0x0007a] in <2e2e6e53129e4ba1874d2ad0eec22c59>:0 at AssemblyUpdater.Application.Program.Main (System.String[] args) [0x00057] in <2e2e6e53129e4ba1874d2ad0eec22c59>:0 UnityEditor.AssetDatabase:Refresh() JetBrains.Rider.Unity.Editor.AdditionalPluginsInstaller:UpdateSelf(String) JetBrains.Platform.RdFramework.Util.Signal1:Fire0(Action1[], String) JetBrains.Platform.RdFramework.Util.Signal1:Fire(String) JetBrains.Platform.RdFramework.Util.RProperty1:set_Value(String) JetBrains.Platform.RdFramework.Impl.<>cDisplayClass21_0:b3(UnsafeReader) JetBrains.Platform.RdFramework.Impl.MessageBroker:Execute(Action`1, RdId, Byte[]) JetBrains.Platform.RdFramework.Impl.<>cDisplayClass9_0:b1() JetBrains.Rider.Unity.Editor.MainThreadDispatcher:DispatchTasks() UnityEditor.EditorApplication:Internal_CallUpdateFunctions()

van800 commented 6 years ago

@trondtactile Please provide some example, which works for you in Unity.

trondtactile commented 6 years ago
using NUnit.Framework;
using Assert = UnityEngine.Assertions.Assert;

public class ExampleTest {

    [Test]
    public void WorkingTest() {
        Assert.IsTrue(true);
    }

    [Test]
    public void NonRunningTest([Values(true, false)] bool value) {
        Assert.IsTrue(value);
    }

}
van800 commented 6 years ago

@trondtactile Thanks! Please expected fix in a couple of days. https://github.com/JetBrains/resharper-unity/commit/29c3dd0c693e7e721d8bdd0c3570c33bd35738ec

trondtactile commented 6 years ago

awesome, thanks!

sdarkwell commented 6 years ago

Hello!

Any status update on this update?

I am using the most recent JetBrains Rider nightly build and am still unable to run NUnit unit tests from Rider itself. They forever display "Pending" until I cancel them, at which time they display "Inconclusive".

Thank you and be well! - S.

quabug commented 6 years ago

also have "pending" problem on the latest nightly building JetBrains Rider 2018.1 EAP Build #RD-181.4952.17, built on May 2, 2018 Unity 2018.1.0f2

trondtactile commented 6 years ago

I also noticed that you still get "unconclusive" if you have the attribute [TestCase()] on the test method. (similar to the [values] attributes)

van800 commented 6 years ago

TestCase, ValueSource and other dynamic tests are in the works https://github.com/JetBrains/resharper-unity/pull/551

van800 commented 6 years ago

Dynamic tests support was added in the latest available nightly build. https://rider-support.jetbrains.com/hc/en-us/articles/360000009804-How-to-get-Rider-Nightly-build-

I am closing this issue in favor of #530

trondtactile commented 6 years ago

Just upgraded to the newest nightly build. Now i can not run non-dynamic tests even. Rider tries to build the application whenever i try to run tests, which fails like so:

Microsoft.Common.CurrentVersion.targets(1601, 5): [MSB4036] The "GetReferenceNearestTargetFrameworkTask" task was not found. Check the following: 1.) The name of the task in the project file is the same as the name of the task class. 2.) The task class is "public" and implements the Microsoft.Build.Framework.ITask interface. 3.) The task is correctly declared with <UsingTask> in the project file, or in the *.tasks files located in the "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\MSBuild\15.0\Bin" directory.

van800 commented 6 years ago

I am sorry for inconvenience. You may try to change Launch - Build Policy to "Never" image Or if it doesn't help then fix MSBuild installation (https://developercommunity.visualstudio.com/content/problem/137779/the-getreferencenearesttargetframeworktask-task-wa.html)

I will try to come up with better solution later today

van800 commented 6 years ago

@trondtactile As far as I can see, if you have selected to run tests via UnityEditor and you have connection to UnityEditor, build is not fired. If the problem reproduces for you, let me see the logs from Rider->Help->Show log in Files, collect all log files please.

image

van800 commented 6 years ago

@trondtactile any update on your last issue? Also please check the nie nightly build with multiple fixes.

igisav commented 6 years ago

@van800 I updated to nightly build #RD-181.4952.153 and still have tests in pending or inconclusive state... Reinstalling plugin, switch between Unity and Rider or recompiling doesn't help. How we can provide more information for you?

trondtactile commented 6 years ago

yeah, tried the latest as well. Disabled building when running tests. Now they are just stuck at pending instead.

van800 commented 6 years ago

Please provide me with your OS, Unity version. And I will check them. If you can spend some time to help, here is what you can do:

  1. Try to switch the dropbox Unity Editor/Standalon NUnit in Rider UnitTesting panel and run in both modes.
  2. Leave it in Unity Editor state.
  3. Goto Unity -> Preferences -> Rider ->Logging level = Trace
  4. Run tests again from Rider. Logs would be placed to %temp%\Unity3dRider folder for Windows and $TMPDIR/Unity3dRider for Mac.
  5. Take the last couple of logs and put here. You may be interested to look for "Error" messages inside.
van800 commented 6 years ago

This issue will be exclusively about running tests via Unity. And #530 will be about running tests without Unity via NUnit.

van800 commented 6 years ago

I am sorry, I found out that build #RD-181.4952.153 haven't got all the fixes. Please wait for the next one.

van800 commented 6 years ago

Related to RIDER-15809 Can't run unit tests (pong-tdd)

van800 commented 6 years ago

New nightly build is out. Please try. I checked Unity versions 5.6.3f1, 2017.4.2f2, 2018.1.0f2, 2018.2.0b4 on Windows.

igisav commented 6 years ago

I tested on Mac the new nightly build and it works. (RD-181.4952.191, macOS 10.12.6, Unity 2018.1.0b12)

Yesterday I tried on Windows older Version with Unity 2018.1.0.b5 and got following error in logs:

RiderPlugin[ERROR]22:42:53.472 Exception while launching Unity Editor tests.
Constructor on type 'UnityEditor.TestTools.TestRunner.EditModeLauncher' not found.
  at System.RuntimeType.CreateInstanceImpl (System.Reflection.BindingFlags bindingAttr, System.Reflection.Binder binder, System.Object[] args, System.Globalization.CultureInfo culture, System.Object[] activationAttributes, System.Threading.StackCrawlMark& stackMark) [0x00213] in <dcaae11f4f674b16b268094ee0049a09>:0 
  at System.Activator.CreateInstance (System.Type type, System.Reflection.BindingFlags bindingAttr, System.Reflection.Binder binder, System.Object[] args, System.Globalization.CultureInfo culture, System.Object[] activationAttributes) [0x000a1] in <dcaae11f4f674b16b268094ee0049a09>:0 
  at System.Activator.CreateInstance (System.Type type, System.Reflection.BindingFlags bindingAttr, System.Reflection.Binder binder, System.Object[] args, System.Globalization.CultureInfo culture) [0x00000] in <dcaae11f4f674b16b268094ee0049a09>:0 
  at JetBrains.Rider.Unity.Editor.UnitTesting.UnityEditorTestLauncher.TryLaunchUnitTests () [0x00337] in <ff0e2bf33edf46c0a2aa82d3d0cf880a>:0 
RECV[TRACE]22:51:47.283 endpoint `UnityEditorPlugin.EditorPluginModel.getUnityEditorState`::(7454109928747411040), taskId=1000465, request = JetBrains.Platform.RdFramework.RdVoid
SEND[TRACE]22:51:47.283 endpoint `UnityEditorPlugin.EditorPluginModel.getUnityEditorState`::(7454109928747411040), response = Success :: Idle
RECV[TRACE]22:51:48.241 endpoint `UnityEditorPlugin.EditorPluginModel.getUnityEditorState`::(7454109928747411040), taskId=1000467, request = JetBrains.Platform.RdFramework.RdVoid
SEND[TRACE]22:51:48.241 endpoint `UnityEditorPlugin.EditorPluginModel.getUnityEditorState`::(7454109928747411040), response = Success :: Idle

But maybe you found already the problem cause and fix it. Thanx a lot.

van800 commented 6 years ago

@igisav yes, I think I fixed that one. Thanks for the feedback.

trondtactile commented 6 years ago

I can confirm this now works for me. Thanks! 👍

lordlycastle commented 5 years ago

I can't get it to work with 2019.1 Rider and Unity 2018.4. It says inconclusive test not run. I can run the same tests from Unity's test runner window and they work as expected. It's annoying I can't from rider and we have far more options for running tests than in Unity. The tests are also very simple and not dependant on any MonoBehaviour or unity specific code. It's testing code of static classes.

Any idea what is causing this?

van800 commented 5 years ago

@lordlycastle If tests are not Unity ones, without access to GameObjects, you can switch TestRunner to use Standalone nUnit. image

van800 commented 5 years ago

@lordlycastle Please check in Unity that your tests are Editor (not Player) ones. I tried 2018.4 with EditorTests and it worked just fine.

lordlycastle commented 5 years ago

@van800 I tried that "Standalone NUnit Launcher". I get the following error:

System.Security.SecurityException : ECall methods must be packaged into a system module.
TearDown : System.Security.SecurityException : ECall methods must be packaged into a system module.
   at UnityEngine.DebugLogHandler.LogFormat(LogType logType, Object context, String format, Object[] args)
   at UnityEngine.Logger.Log(LogType logType, Object message)
   at *** MY TEST FILE.cs ***
--TearDown
   at UnityEngine.DebugLogHandler.LogFormat(LogType logType, Object context, String format, Object[] args)
   at UnityEngine.Logger.Log(LogType logType, Object message)
   at *** MY TEST FILE.cs ***

However, we will add some unity tests later and will require to use Unity Editor mode.

lordlycastle commented 5 years ago

@van800 I don't understand what you mean by "Editor not Player".

van800 commented 5 years ago

@lordlycastle you may check in Unity test runner image

van800 commented 5 years ago

ECall methods must be packaged into a system module means tests are calling Unity native API-s, which can't be called in standalone nunit runner, only in real Unity environment.

lordlycastle commented 5 years ago

@van800 Yes it in Edit mode and tests show up there and run fine.

You're right, I was using a Debug.Log in the tear down. Is there a preprocessor flag that can be use to switch between Stanalone Nunit and Unity Editor

van800 commented 5 years ago

@lordlycastle This would work:

            try
            {
                Debug.Log("test");
            }
            catch
            {
                Console.WriteLine("test");
            }
van800 commented 5 years ago

@lordlycastle I am curious to see what is causing the issue, when running in UnityEditor mode from Rider.

Could you send me some sample project to Ivan.Shakhov@jetbrains.com? Other option is to Enable Trace logs in Unity Preferences Rider, reproduce the issue and send me all logs from the folder %Temp%\Unity3dRider Thanks!

lordlycastle commented 5 years ago

@van800 I've sent you a test project with the issue at hand. Thanks for the help.

van800 commented 5 years ago

We had a chat with @lordlycastle about his case with TestCaseSource.

        public struct MultiplyTestCase
        {
            public int result;
            public (int, int) input;
        }

        public int Multiply(int x, int y) => x * y;

        private static readonly List<MultiplyTestCase> _testCases =
                new List<MultiplyTestCase>
                {
                        new MultiplyTestCase {result = 4, input = (2, 2),},
                        // Fail test case.
                        new MultiplyTestCase {result = 5, input = (2, 3),},
                        // Success test case.
                        new MultiplyTestCase {result = 9, input = (3, 3),}
                };

        [Test, TestCaseSource(nameof(_testCases))]
        public void TestMultiply(MultiplyTestCase testCase)
        {
            Assert.AreEqual(testCase.result, Multiply(testCase.input.Item1, testCase.input.Item2));
        }

In Unity it produces 3 tests with similar names: image But for Rider it displays only one test. Workaround is to override a ToString() in MultiplyTestCase:

public override string ToString()
{
   return input + "=" + result;
}

In general I believe we can't do much with it from Rider side, it requires a fix in Unity.