search

benaclejames / VRCFaceTracking

OSC App to allow VRChat avatars to interact with eye and facial tracking hardware
https://docs.vrcft.io
Apache License 2.0
615 stars 96 forks source link

Crashed in some avatars and worlds #46

Closed ShawnMayer closed 3 years ago

ShawnMayer commented 3 years ago

I found crashes in some avatars and worlds while using this mod. It used to happen very infrequently before, but after this 2019 Unity update, it's happened very often.

Error message is "Fatal error in GC / Collecting from unknown thread" unknown

Sample World : [https://vrchat.com/home/world/wrld_95898adc-bf3d-42fb-b82b-8a4cd5a8f25f]

Maybe if you using VRCFaceTracking, you can't be in this world for more than an hour. (If you have someone else's avatar, symptoms will appear more quickly and may not last even 3 minutes.)

I've been testing all day to isolate this issue and have confirmed that the crash doesn't occur with the mod removed.

benaclejames commented 3 years ago

I'm aware of the issue and somewhat confused as to what causes it haha. I'm working on a fix but I'm not sure how long it'll be until I fix it

MoePus commented 3 years ago

a work around for current version: patch GameAssembly.dll+0x1559B5 75 -> EB

No crash for an 8 hours continuously play

Edit 20210813: GameAssembly.dll+0x13DA35 75 -> EB

howard1414 commented 3 years ago

a work around for current version: patch GameAssembly.dll+0x1559B5 75 -> EB

No crash for an 8 hours continuously play

In the GameAssembly.dll+0x1559B5 , did not saw the value of "75". Which I got here is the "FE" Value , am I jumping to the wrong address ?

image

MoePus commented 3 years ago

a work around for current version: patch GameAssembly.dll+0x1559B5 75 -> EB No crash for an 8 hours continuously play

In the GameAssembly.dll+0x1559B5 , did not saw the value of "75". Which I got here is the "FE" Value , am I jumping to the wrong address ?

image

Its RVA. patch it dynamically with CheatEngine

howard1414 commented 3 years ago

a work around for current version: patch GameAssembly.dll+0x1559B5 75 -> EB No crash for an 8 hours continuously play

In the GameAssembly.dll+0x1559B5 , did not saw the value of "75". Which I got here is the "FE" Value , am I jumping to the wrong address ? image

Its RVA. patch it dynamically with CheatEngine

Edit2: After testing about 8 hours , haven't got crash yet , this temporary solution works great!

Edit: Finally figure out how to edit the RVA address in CheatEngine , I'll tested out this night if the problem can be fix by this method temporary. image

ShawnMayer commented 3 years ago

a work around for current version: patch GameAssembly.dll+0x1559B5 75 -> EB

No crash for an 8 hours continuously play

I tired this and works great!!! Thank you so much 👍

benaclejames commented 3 years ago

a work around for current version: patch GameAssembly.dll+0x1559B5 75 -> EB

No crash for an 8 hours continuously play

Hey, just wondering what this actually does behind the scenes so I can better understand how to implement it. Thanks

MoePus commented 3 years ago

a work around for current version: patch GameAssembly.dll+0x1559B5 75 -> EB No crash for an 8 hours continuously play

Hey, just wondering what this actually does behind the scenes so I can better understand how to implement it. Thanks

I dont know clearly either. Mono(?or something il2cpped) register threads create by itself as known thread.(saves the thread id in a data structure) When gc, it checks object's thread.If the thread is not a known thread, it triggers the int3 trapper.

I just patched the check. When creating new threads, it ignores checking known thread too so i think its not so harmful to bypass this check.

Maybe use threading api provided by the game could avoid this problem,but i dont know

200Tigersbloxed commented 3 years ago

Did some digging, and I have a guess at where the issue is.

Reference 1) https://github.com/benaclejames/VRCFaceTracking/blob/master/VRCFaceTracking/UnifiedLibManager.cs#L28 Reference 2) https://github.com/benaclejames/VRCFaceTracking/blob/master/VRCFaceTracking/UnifiedLibManager.cs#L35

In Reference 1, I see the method is being called on VRC_OnUiManagerInit(), which if I'm correct (which I probably am not), then this calls everytime some UI manager in vrchat is loaded. I believe this Reference also initializes whatever SDK you're using (SRanipal/Pimax), which is where I think the issue is here. Rather than started the runtimes on UI manager init, I think they should be called OnApplicationStart(), this way, we can initialize the SDK first, then just pull the data from whatever static variable/method when necessary.

In Reference 2, we have what looks to be the method to find the runtime you're using Eye tracking with. I don't see anything wrong here, mainly because it doesn't create any new Threads, which is 9/10 what the issue is, since it's an Unknown Thread issue. I do however see that there's a SceneManager reference, which I think should be moved into it's on separate OnSceneLoaded() event listener and add some bool(s) for if-conditions, rather than have it inside of Reference 2.

Once again, this is only a guess at what the issue is, although I'm fairly confident that this is an issue with the creation/abortion of Threads. Another possible solution is maybe have a loop in the thread so that we know it doesn't Abort randomly, and when Abort() gets called, it doesn't error and whatnot.

benaclejames commented 3 years ago

Using Moe's advice on threading, I've released a test build using the il2cpp_thread_attach method in an attempt to register the thread with IL2CPP's GC. Hopefully this fixes it but I haven't had a ton of time to test it. https://github.com/benaclejames/VRCFaceTracking/releases/tag/v2.1.6

200Tigersbloxed commented 3 years ago

Latest PR doesn't fix it, crashed almost instantly after joining a new world. Here's how to reproduce it fast though. Open your SDK3 Avatar debug menu and you'll have greater chances of crashing.

Note: I'm using a modified build of VRCFaceTracking for CombinedEyeLid variable (cause it got removed in the latest Pull)

Edit: Judging by the fact that I only crash when the mod uses SRanipal, it must be some thread in the SRanipalTrackingInterface.cs that does it. I've been messing around with it, but not too much luck until I replaced Threads with MelonCoroutines (for fun, cause the frames were way worse on MelonCoroutines) and only that stopped it.

bizzclaw commented 3 years ago

I still got the garbage collection crash yesterday with the latest patch in the releases after about 5 minutes. It always seems to happen a few minutes after I open my debug menu and testing on Spirit of the Sea.

Thank you for all your hard work trying to get this working despite the weird issue seemingly cropping out of nowhere.

Weegee76 commented 3 years ago

Having the same problem, GC is collecting from unknown thread. Happens at varying times with the Vive Pro Eye.

a work around for current version: patch GameAssembly.dll+0x1559B5 75 -> EB

No crash for an 8 hours continuously play

Edit 20210813: GameAssembly.dll+0x13DA35 75 -> EB

How would I implement this workaround? I'm unfamiliar with Cheat Engine in this way.

benaclejames commented 3 years ago

Would it be possible to run MelonLoader in debug mode using the launch option --melonloader.debug next time you crash. It might give some insight as to why this is still happening. Still, weird since a couple of friends and I were able to use it for days at a time with no crash after the fix.

shukaku7 commented 3 years ago

You actually don't have to use Cheat Engine to do the workaround. You can just open GameAssembly.dll in HxD, do a Search for Hex string 45 85 ED 75 17 and change the 75 -> EB then save the file.

InsaneGrox commented 3 years ago

weirdly enough I don't have the GC issue anymore despite it "not being fixed" according to everyone... maybe the fix made it happen a lot less but didn't completely fix it? either way it's an improvement if that's the case

Weegee76 commented 3 years ago

You actually don't have to use Cheat Engine to do the workaround. You can just open GameAssembly.dll in HxD, do a Search for Hex string 45 85 ED 75 17 and change the 75 -> EB then save the file.

Thanks man. But now that I'm hearing the fix is starting to work I'll try it first.

ShawnMayer commented 3 years ago

I've been testing fix version for about 4 days and no more crashes. Sincere thanks to Ben!

bizzclaw commented 3 years ago

I'm not getting the crash anymore on the latest version and now everything seems to be working properly. Thank you!

benaclejames commented 3 years ago

Judging by the number of responses saying the latest release fixed this, I'm closing this thread. Please feel free to continue conversation in here if the issue persists. Thanks for everyone's input!