Frozenreflex
commented
2 years ago I've been trying to debug this (at least my instance of the issue, random display freezes + crashes/unresponsive engine messages) by using mods for months now, and I've gotten more information about the issue, but I still don't know exactly where the code is getting locked up at. Due to the structure of Unity's code, it is shockingly difficult to get stack traces from arbitrary threads, and I don't think I could do much more to try and find the cause of the issue at this rate. I added this to GetStackTrace
[HarmonyPatch(typeof(FrooxEngine.Engine))]
class EnginePatch
{
[HarmonyPrefix]
[HarmonyPatch("GetStackTrace")]
public static void GetStackTrace(Thread targetThread)
{
UniLog.Log("Thread state: " + targetThread.ThreadState);
targetThread.Abort();
}
}to try and get the main thread to throw an exception, since a good chunk of the code is encapsulated with try/catch and log printing statements. However, potentially due to one of two different issues, this doesn't work. Subsequent calls will show that the thread's state is AbortRequested, but it never fully aborts, and gets stuck like this, never throwing the exception. According to Microsoft's documentation, if a thread is in unmanaged (c++) code, the thread won't abort until execution is returned to the managed thread. However, I also found that Abort may not function properly in Unity, but the threads that did say this were using older versions of Unity, and I couldn't find anything more recent (and nothing about what the thread's state gets set to). This could mean that execution is getting stuck in unmanaged code, but it could also mean nothing. In addition to trying this, I've been manually adding logic to track code flow, and can eliminate these functions from causing (at least my instance of) the issue:
FrooxEngineRunner.Update (surprisingly, this finishes normally, but because of Unity logic shenanigans, stops being called, which is the issue)
CameraRenderEx.OnPreCull
CameraRenderEx.OnPreRender
CameraRenderEx.OnPostRender
OverlayRootPositioner.LateUpdate
ParticleSystemBehavior.LateUpdate
ParticleSystemBehavior.UpdateStyle (never called)
CameraConnector.ApplyChanges
AudioOutputConnector.ApplyChanges
BlitToDisplayConnector.ApplyChanges (never called)
CameraPortalConnector.ApplyChanges
LightConnector.ApplyChanges
LODGroupConnector.ApplyChanges (never called)
MaterialConnector.ApplyChanges
RenderQueueProcessor.LateUpdate
This was done by tracking when the function starts and ends using prefixes and postfixes, and theoretically any functions used within are also not the cause (with exception to those never called). While I've been able to rule out these, doing so is time consuming and a nightmare to implement, and adding the logging statements I used to the entire codebase using patches is basically impossible, which is why I don't think there's much further I can go here.
I guess it's also important to note that I only encounter this crash/freeze in VR, desktop never has this issue for me, so it could potentially be something that both VR and headless sessions (Enverex's logs) use, but not desktop. It could also be two different, potentially similar issues that cause similar symptoms, and I can't check headless sessions since I don't have access to them.
orange3134
Describe the bug?
Neos BLFC world, bunch of user's neos starts hanging but not for everyone. On my headless it's happened as well. I'll be supplying everyone's log that was present. We're not 100% certain what exactly is causing it. There doesn't seem to be a common occurance. I did notice twice when I went into free cam mode, everyone instantly got a world crash and had to rejoin.
Relevant issues
Possibly #3244 ??
To Reproduce
Like I mentioned before, I have no idea how to reproduce hence supplying a bunch of user's logs.
Expected behavior
For Neos to not hang without a crash error.
Log Files
Mine headless's Logs: DESKTOP-SPEUN1P - 2021.10.26.9 - 2021-10-25 21_14_10.log
My User Log:
[DESKTOP-SPEUN1P - 2021.10.26.9 - 2021-10-28 00_2700.log](https://github.com/Neos-Metaverse/NeosPublic/files/7431892/DESKTOP-SPEUN1P.-.2021.10.26.9.-.2021-10-28.00
DESKTOP-SPEUN1P - 2021.10.26.9 - 2021-10-28 02_47_08.log 27_00.log)
My Player Logs: Player.log Player-prev.log
My logs when my game hung in BLFC pool world: Player.log DESKTOP-SPEUN1P_-2021.10.26.9-_2021-10-27_21_12_08.log
Soap's Log:
DESKTOP-M3LTEEK_-2021.10.26.9-_2021-10-27_21_03_13.log
Screenshots
No response
How often does it happen?
Sometimes
Does the bug persist after restarting Neos?
Yes
Neos Version Number
2021.10.26.9
What Platforms does this occur on?
Windows
Link to Reproduction Item/World
No response
Did this work before?
No
If it worked before, on which build?
No response
Additional context
If anyone else has log files of when Neos hangs and doesn't return or you suddenly get a world crash error. Please contribute some logs!
Your player log can be found:
\AppData\LocalLow\Solirax\NeosVRYour normal log can be found:\steamapps\common\NeosVR\LogsReporters
AshtonSparx#5050 AdmiralSoap#4934