search

alttester / AltTester-Unity-SDK

AltTester Unity SDK is an open-source UI driven test automation tool that helps you find objects in your game and interacts with them using tests written in C#, Python, Robot or Java.
https://alttester.com/docs/sdk/latest/home.html
GNU General Public License v3.0
63 stars 7 forks source link

Unity Freezes When Running Tests In Editor #70

Closed AltTesterBot closed 2 years ago

AltTesterBot commented 2 years ago

Hey folks,

I want to start by saying this works great for my needs on Android devices (haven't tried yet on iOS) and I'm having a blast running tests on my phone.

However, I have a few people who don't want to have to build when they make game code changes to see if it's still operating correctly and they'd like to run the tests in the editor. Problem is, the Editor will freeze and never restore functionality (we have waited 10 minutes before). I'm then forced to kill the process.

I could not find any documentation on how to properly run tests in the editor, so this is how I've been trying it: Window -> AltUnityTester -> Run in Editor -> Wait for moment that tests should be run -> RunSelectedTest

I've tried it with "Debug Build Needed" checked and unchecked, and it always gets to the same point.

My test is just an initial temporary test that hits the play button:

Debug.Log("Starting Play button tap test"); AltUnityDriver.FindElement("PlayButton").Tap();

Here's the Editor.log from start of test to the last point it logged:


UnityEngine.DebugLogHandler:Internal_Log(LogType, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
AltUnityTesterEditor:RunTests(TestRunMode) (at Assets\AltUnityTester\AltUnityDriver\Editor\AltUnityTesterEditor.cs:94)
AltUnityTesterEditor:OnGUI() (at Assets\AltUnityTester\AltUnityDriver\Editor\AltUnityTesterEditor.cs:512)
System.Reflection.MonoMethod:InternalInvoke(Object, Object[], Exception&)
System.Reflection.MonoMethod:Invoke(Object, BindingFlags, Binder, Object[], CultureInfo) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:222)
System.Reflection.MethodBase:Invoke(Object, Object[]) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MethodBase.cs:115)
UnityEditor.HostView:Invoke(String, Object) (at C:\buildslave\unity\build\Editor\Mono\HostView.cs:295)
UnityEditor.HostView:Invoke(String) (at C:\buildslave\unity\build\Editor\Mono\HostView.cs:288)
UnityEditor.HostView:InvokeOnGUI(Rect) (at C:\buildslave\unity\build\Editor\Mono\HostView.cs:255)
UnityEditor.DockArea:OldOnGUI() (at C:\buildslave\unity\build\Editor\Mono\GUI\DockArea.cs:390)
UnityEngine.Experimental.UIElements.IMGUIContainer:DoOnGUI(Event, Boolean) (at C:\buildslave\unity\build\Modules\UIElements\IMGUIContainer.cs:215)
UnityEngine.Experimental.UIElements.IMGUIContainer:HandleIMGUIEvent(Event) (at C:\buildslave\unity\build\Modules\UIElements\IMGUIContainer.cs:380)
UnityEngine.Experimental.UIElements.IMGUIContainer:HandleEvent(EventBase) (at C:\buildslave\unity\build\Modules\UIElements\IMGUIContainer.cs:359)
UnityEngine.Experimental.UIElements.EventDispatcher:DispatchEvent(EventBase, IPanel) (at C:\buildslave\unity\build\Modules\UIElements\EventDispatcher.cs:260)
UnityEngine.Experimental.UIElements.UIElementsUtility:DoDispatch(BaseVisualElementPanel) (at C:\buildslave\unity\build\Modules\UIElements\UIElementsUtility.cs:243)
UnityEngine.Experimental.UIElements.UIElementsUtility:ProcessEvent(Int32, IntPtr) (at C:\buildslave\unity\build\Modules\UIElements\UIElementsUtility.cs:74)
UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr) (at C:\buildslave\unity\build\Modules\IMGUI\GUIUtility.cs:171)

(Filename: Assets/AltUnityTester/AltUnityDriver/Editor/AltUnityTesterEditor.cs Line: 94)

Accept thread id: 2
UnityEngine.DebugLogHandler:Internal_Log(LogType, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
AltSocketServer:AcceptCallback(IAsyncResult) (at Assets\AltUnityTester\AltUnityServer\AltSocket\AltSocketServer.cs:119)
System.Net.Sockets.SocketAsyncResult:Complete()
System.Net.Sockets.SocketAsyncResult:Complete(Socket)
System.Net.Sockets.Worker:Accept()

(Filename: Assets/AltUnityTester/AltUnityServer/AltSocket/AltSocketServer.cs Line: 119)

Starting Play button tap test
UnityEngine.DebugLogHandler:Internal_Log(LogType, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
SmokeTests:InitialTest() (at Assets\Scripts\Editor\Tests\SmokeTests.cs:23)
System.Reflection.MonoMethod:InternalInvoke(Object, Object[], Exception&)
System.Reflection.MonoMethod:Invoke(Object, BindingFlags, Binder, Object[], CultureInfo) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MonoMethod.cs:222)
System.Reflection.MethodBase:Invoke(Object, Object[]) (at /Users/builduser/buildslave/mono/build/mcs/class/corlib/System.Reflection/MethodBase.cs:115)
NUnit.Framework.Internal.Reflect:InvokeMethod(MethodInfo, Object, Object[])
NUnit.Framework.Internal.MethodWrapper:Invoke(Object, Object[])
NUnit.Framework.Internal.Commands.TestMethodCommand:RunNonAsyncTestMethod(ITestExecutionContext)
NUnit.Framework.Internal.Commands.TestMethodCommand:RunTestMethod(ITestExecutionContext)
NUnit.Framework.Internal.Commands.TestMethodCommand:Execute(ITestExecutionContext)
NUnit.Framework.Internal.Commands.TestActionCommand:Execute(ITestExecutionContext)
NUnit.Framework.Internal.Commands.SetUpTearDownCommand:Execute(ITestExecutionContext)
NUnit.Framework.Internal.Execution.SimpleWorkItem:PerformWork()
NUnit.Framework.Internal.Execution.WorkItem:RunTest()
NUnit.Framework.Internal.Execution.WorkItem:Execute()
NUnit.Framework.Internal.Execution.SimpleWorkItemDispatcher:Dispatch(WorkItem)
NUnit.Framework.Internal.Execution.CompositeWorkItem:RunChildren()
NUnit.Framework.Internal.Execution.CompositeWorkItem:PerformWork()
NUnit.Framework.Internal.Execution.WorkItem:RunTest()
NUnit.Framework.Internal.Execution.WorkItem:Execute()
NUnit.Framework.Internal.Execution.SimpleWorkItemDispatcher:Dispatch(WorkItem)
NUnit.Framework.Internal.Execution.CompositeWorkItem:RunChildren()
NUnit.Framework.Internal.Execution.CompositeWorkItem:PerformWork()
NUnit.Framework.Internal.Execution.WorkItem:RunTest()
NUnit.Framework.Internal.Execution.WorkItem:Execute()
NUnit.Framework.Internal.Execution.SimpleWorkItemDispatcher:RunnerThreadProc()

(Filename: Assets/Scripts/Editor/Tests/SmokeTests.cs Line: 23)

thread id 2 accepted client 127.0.0.1:62136
UnityEngine.DebugLogHandler:Internal_Log(LogType, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
AltSocketServer:AcceptCallback(IAsyncResult) (at Assets\AltUnityTester\AltUnityServer\AltSocket\AltSocketServer.cs:123)
System.Net.Sockets.SocketAsyncResult:Complete()
System.Net.Sockets.SocketAsyncResult:Complete(Socket)
System.Net.Sockets.Worker:Accept()

(Filename: Assets/AltUnityTester/AltUnityServer/AltSocket/AltSocketServer.cs Line: 123)

thread id 2 beginning read from client 127.0.0.1:62136
UnityEngine.DebugLogHandler:Internal_Log(LogType, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
AltSocketServer:AcceptCallback(IAsyncResult) (at Assets\AltUnityTester\AltUnityServer\AltSocket\AltSocketServer.cs:124)
System.Net.Sockets.SocketAsyncResult:Complete()
System.Net.Sockets.SocketAsyncResult:Complete(Socket)
System.Net.Sockets.Worker:Accept()

(Filename: Assets/AltUnityTester/AltUnityServer/AltSocket/AltSocketServer.cs Line: 124)

Client thread started
UnityEngine.DebugLogHandler:Internal_Log(LogType, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
AltSocketServer:AcceptCallback(IAsyncResult) (at Assets\AltUnityTester\AltUnityServer\AltSocket\AltSocketServer.cs:135)
System.Net.Sockets.SocketAsyncResult:Complete()
System.Net.Sockets.SocketAsyncResult:Complete(Socket)
System.Net.Sockets.Worker:Accept()

(Filename: Assets/AltUnityTester/AltUnityServer/AltSocket/AltSocketServer.cs Line: 135)

Max number of clients reached (1), stopping listening
UnityEngine.DebugLogHandler:Internal_Log(LogType, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
AltSocketServer:AcceptCallback(IAsyncResult) (at Assets\AltUnityTester\AltUnityServer\AltSocket\AltSocketServer.cs:141)
System.Net.Sockets.SocketAsyncResult:Complete()
System.Net.Sockets.SocketAsyncResult:Complete(Socket)
System.Net.Sockets.Worker:Accept()

(Filename: Assets/AltUnityTester/AltUnityServer/AltSocket/AltSocketServer.cs Line: 141)

find object by name PlayButton
UnityEngine.DebugLogHandler:Internal_Log(LogType, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
AltUnityRunner:ClientSocketHandlerDidReadMessage(AltClientSocketHandler, String) (at Assets\AltUnityTester\AltUnityServer\AltUnityRunner.cs:232)
AltClientSocketHandler:Run() (at Assets\AltUnityTester\AltUnityServer\AltSocket\AltClientSocketHandler.cs:78)

(Filename: Assets/AltUnityTester/AltUnityServer/AltUnityRunner.cs Line: 232)

Stopped listening for connections
UnityEngine.DebugLogHandler:Internal_Log(LogType, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
AltSocketServer:StopListeningForConnections() (at Assets\AltUnityTester\AltUnityServer\AltSocket\AltSocketServer.cs:148)
AltSocketServer:AcceptCallback(IAsyncResult) (at Assets\AltUnityTester\AltUnityServer\AltSocket\AltSocketServer.cs:142)
System.Net.Sockets.SocketAsyncResult:Complete()
System.Net.Sockets.SocketAsyncResult:Complete(Socket)
System.Net.Sockets.Worker:Accept()

(Filename: Assets/AltUnityTester/AltUnityServer/AltSocket/AltSocketServer.cs Line: 148)

4
UnityEngine.DebugLogHandler:Internal_Log(LogType, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:Log(LogType, Object)
UnityEngine.Debug:Log(Object)
AltUnityRunner:ClientSocketHandlerDidReadMessage(AltClientSocketHandler, String) (at Assets\AltUnityTester\AltUnityServer\AltUnityRunner.cs:233)
AltClientSocketHandler:Run() (at Assets\AltUnityTester\AltUnityServer\AltSocket\AltClientSocketHandler.cs:78)

(Filename: Assets/AltUnityTester/AltUnityServer/AltUnityRunner.cs Line: 233)```

---
<sub>You can find the original issue from GitLab [here](https://gitlab.com/altom/altunity/altunitytester/-/issues/75).</sub>
AltTesterBot commented 2 years ago

Hi, we know about this problem. I think the problem is that both the AltUnityTester and the game are using the same thread and freezes when AltUnityTester is waiting for a response.

A workaround to run the test would be to make a new c# project and copy every script from AltUnityDriver folder without the Editor folder(you can copy it but it is not necessary) and the test script and run the test from there.

We will try to fix as soon as possible but until then you could use the workaround.

You can find the original comment from GitLab here.