search

CesiumGS / cesium-unity

Bringing the 3D geospatial ecosystem to Unity
https://cesium.com/platform/cesium-for-unity/
Apache License 2.0
343 stars 81 forks source link

Crash on Quest 2 / Quest 3 after around 20 minutes of playtime #408

Open electrum-bowie opened 7 months ago

electrum-bowie commented 7 months ago

Cesium for Unity 1.7.1.

I should also add that it only happens in a Scene with heavier script processing (multiple runtime behaviour scripts running at the same time), What do you suggest i do? Maybe lowering the Max Cached Bytes property? is Memory Even the issue??

16 people using a quest vr device (android) experienced this exact crash on their own device atleast once: Crash Reason Abort Reason: terminating (libCesiumForUnityNative-Runtime.so)

Thread 25382

00 pc 000000000008ada0 /apex/com.android.runtime/lib64/bionic/libc.so (abort+168) (BreakpadBuildId: 06550e2bcf68e781af8072edf3aa70cf0)

01 pc 00000000018c4648 libil2cpp.so (libil2cpp.so (0x18c4648)) (BreakpadBuildId: 05ff99529c6fff0900000000000000000)

02 pc 00000000018c4798 libil2cpp.so (libil2cpp.so (0x18c4798)) (BreakpadBuildId: 05ff99529c6fff0900000000000000000)

03 pc 00000000018c4700 libil2cpp.so (libil2cpp.so (0x18c4700)) (BreakpadBuildId: 05ff99529c6fff0900000000000000000)

04 pc 00000000018c4684 libil2cpp.so (std::terminate()+36) (BreakpadBuildId: 05ff99529c6fff0900000000000000000)

05 pc 00000000018c48a8 libil2cpp.so (libil2cpp.so (0x18c48a8)) (BreakpadBuildId: 05ff99529c6fff0900000000000000000)

06 pc 0000000000a65178 /data/app/~~ImIKlHoLSZzYNYi4ko35hQ==/com.bowie.earthquest-wruUPKBVmawpC_Ev8jOzRQ==/lib/arm64/libCesiumForUnityNative-Runtime.so (std::__terminate(void ()())+12)

07 pc 0000000000a659b8 /data/app/~~ImIKlHoLSZzYNYi4ko35hQ==/com.bowie.earthquest-wruUPKBVmawpC_Ev8jOzRQ==/lib/arm64/libCesiumForUnityNative-Runtime.so (__cxxabiv1::call_terminate(bool, _Unwind_Exception)+36)

08 pc 0000000000a6595c /data/app/~~ImIKlHoLSZzYNYi4ko35hQ==/com.bowie.earthquest-wruUPKBVmawpC_Ev8jOzRQ==/lib/arm64/libCesiumForUnityNative-Runtime.so (cxxabiv1::scan_eh_tab(cxxabiv1::(anonymous namespace)::scan_results&, _Unwind_Action, bool, _Unwind_Exception, _Unwind_Context)+1508)

09 pc 0000000000a65240 /data/app/~~ImIKlHoLSZzYNYi4ko35hQ==/com.bowie.earthquest-wruUPKBVmawpC_Ev8jOzRQ==/lib/arm64/libCesiumForUnityNative-Runtime.so (__gxx_personality_v0+104)

10 pc 00000000018dbb38 libil2cpp.so (libil2cpp.so (0x18dbb38)) (BreakpadBuildId: 05ff99529c6fff0900000000000000000)

11 pc 00000000018c3bf0 libil2cpp.so (__cxa_throw+116) (BreakpadBuildId: 05ff99529c6fff0900000000000000000)

12 pc 0000000001842e0c libil2cpp.so (libil2cpp.so (0x1842e0c)) (BreakpadBuildId: 05ff99529c6fff0900000000000000000)

13 pc 00000000017fb94c libil2cpp.so (libil2cpp.so (0x17fb94c)) (BreakpadBuildId: 05ff99529c6fff0900000000000000000)

14 pc 00000000017fbd70 libil2cpp.so (libil2cpp.so (0x17fbd70)) (BreakpadBuildId: 05ff99529c6fff0900000000000000000)

15 pc 00000000019bb66c libil2cpp.so (libil2cpp.so (0x19bb66c)) (BreakpadBuildId: 05ff99529c6fff0900000000000000000)

16 pc 00000000015d1f54 libil2cpp.so (libil2cpp.so (0x15d1f54)) (BreakpadBuildId: 05ff99529c6fff0900000000000000000)

17 pc 000000000044cc98 /data/app/~~ImIKlHoLSZzYNYi4ko35hQ==/com.bowie.earthquest-wruUPKBVmawpC_Ev8jOzRQ==/lib/arm64/libCesiumForUnityNative-Runtime.so (CesiumForUnityNative::Cesium3DTilesetImpl::Update(DotNet::CesiumForUnity::Cesium3DTileset const&)+356)

18 pc 0000000000425a24 /data/app/~~ImIKlHoLSZzYNYi4ko35hQ==/com.bowie.earthquest-wruUPKBVmawpC_Ev8jOzRQ==/lib/arm64/libCesiumForUnityNative-Runtime.so (DotNet_CesiumForUnity_Cesium3DTileset_Update+60)

19 pc 0000000001a29974 libil2cpp.so (libil2cpp.so (0x1a29974)) (BreakpadBuildId: 05ff99529c6fff0900000000000000000)

20 pc 0000000001859468 libil2cpp.so (libil2cpp.so (0x1859468)) (BreakpadBuildId: 05ff99529c6fff0900000000000000000)

21 pc 00000000018593b4 libil2cpp.so (libil2cpp.so (0x18593b4)) (BreakpadBuildId: 05ff99529c6fff0900000000000000000)

22 pc 000000000063ef90 libunity.so (libunity.so (0x63ef90)) (BreakpadBuildId: 467edd95c85b31b500000000000000000)

23 pc 000000000064e8cc libunity.so (libunity.so (0x64e8cc)) (BreakpadBuildId: 467edd95c85b31b500000000000000000)

24 pc 000000000065ca50 libunity.so (libunity.so (0x65ca50)) (BreakpadBuildId: 467edd95c85b31b500000000000000000)

25 pc 0000000000484308 libunity.so (libunity.so (0x484308)) (BreakpadBuildId: 467edd95c85b31b500000000000000000)

26 pc 00000000005491d0 libunity.so (libunity.so (0x5491d0)) (BreakpadBuildId: 467edd95c85b31b500000000000000000)

27 pc 0000000000549210 libunity.so (libunity.so (0x549210)) (BreakpadBuildId: 467edd95c85b31b500000000000000000)

28 pc 000000000054949c libunity.so (libunity.so (0x54949c)) (BreakpadBuildId: 467edd95c85b31b500000000000000000)

29 pc 00000000006c0ef8 libunity.so (libunity.so (0x6c0ef8)) (BreakpadBuildId: 467edd95c85b31b500000000000000000)

30 pc 00000000006dd318 libunity.so (libunity.so (0x6dd318)) (BreakpadBuildId: 467edd95c85b31b500000000000000000)

31 pc 0000000000004e0c /data/app/~~ImIKlHoLSZzYNYi4ko35hQ==/com.bowie.earthquest-wruUPKBVmawpC_Ev8jOzRQ==/oat/arm64/base.odex (art_jni_trampoline+92)

32 pc 0000000000008634 /data/app/~~ImIKlHoLSZzYNYi4ko35hQ==/com.bowie.earthquest-wruUPKBVmawpC_Ev8jOzRQ==/oat/arm64/base.odex (com.unity3d.player.L.handleMessage+692)

33 pc 00000000000c84e0 /system/framework/arm64/boot-framework.oat (android.os.Handler.dispatchMessage+144) (BreakpadBuildId: b4cde2d562dfd48c80445adfd718ab400)

34 pc 00000000000c9700 /system/framework/arm64/boot-framework.oat (android.os.Looper.loopOnce+1424) (BreakpadBuildId: b4cde2d562dfd48c80445adfd718ab400)

35 pc 00000000000c90d4 /system/framework/arm64/boot-framework.oat (android.os.Looper.loop+516) (BreakpadBuildId: b4cde2d562dfd48c80445adfd718ab400)

36 pc 00000000000094c4 /data/app/~~ImIKlHoLSZzYNYi4ko35hQ==/com.bowie.earthquest-wruUPKBVmawpC_Ev8jOzRQ==/oat/arm64/base.odex (com.unity3d.player.O.run+244)

37 pc 0000000000218964 /apex/com.android.art/lib64/libart.so (art_quick_invoke_stub+548) (BreakpadBuildId: c3d95a7d77dcec225dce96800a7a95940)

38 pc 000000000028495c /apex/com.android.art/lib64/libart.so (art::ArtMethod::Invoke(art::Thread, unsigned int, unsigned int, art::JValue, char const)+188) (BreakpadBuildId: c3d95a7d77dcec225dce96800a7a95940)

39 pc 0000000000628a7c /apex/com.android.art/lib64/libart.so (art::JValue art::InvokeVirtualOrInterfaceWithJValuesart::ArtMethod(art::ScopedObjectAccessAlreadyRunnable const&, _jobject, art::ArtMethod, jvalue const)+460) (BreakpadBuildId: c3d95a7d77dcec225dce96800a7a95940)

40 pc 00000000006786fc /apex/com.android.art/lib64/libart.so (art::thread::CreateCallback(void*)+1164) (BreakpadBuildId: c3d95a7d77dcec225dce96800a7a95940)

41 pc 00000000000dfdb0 /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+132) (BreakpadBuildId: 06550e2bcf68e781af8072edf3aa70cf0)

42 pc 000000000008c3dc /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64) (BreakpadBuildId: 06550e2bcf68e781af8072edf3aa70cf0)

electrum-bowie commented 7 months ago

Update: i am using Google's P3DT, i have 336870912 allocated bytes in the faulty scene, i optimized some scripts and updated some packages and the crash seems to occur more rarely than before, i also found that i had Incremental Garbage Collector TURNED OFF. which i turned on right after. i will update on this tommorow or the day after.

csciguy8 commented 7 months ago

Very likely related to https://github.com/CesiumGS/cesium-native/issues/739

kring commented 7 months ago

It's hard to say what's going on here. The issue @csciguy8 linked could indeed be the problem, though in that case you should see signs the device ran out of memory. It seems we're in CesiumForUnityNative::Cesium3DTilesetImpl::Update and then "something" goes wrong and there's a crash, but the call stack in between is not very useful so it's very hard to tell what happened. Even the 356 listed doesn't make much sense as a line number, so I'm assuming that's a byte offset in the built code and I don't have an easy way to resolve that to specific code. Can you try running a debug build on the device and see if it provides more useful information?

electrum-bowie commented 5 months ago

It's hard to say what's going on here. The issue @csciguy8 linked could indeed be the problem, though in that case you should see signs the device ran out of memory.

The device does not run out of memory during or before the crash, not even close actually, in fact the device's system software shows a notification that makes me sure the crash was because the headset was out of available memory, if that was the case. (which I previously ran into months ago)

electrum-bowie commented 5 months ago

Can you try running a debug build on the device and see if it provides more useful information?

Will do that when I get the time.

electrum-bowie commented 5 months ago

Best possible workaround for now 😁

https://github.com/CesiumGS/cesium-unity/issues/447

csciguy8 commented 5 months ago

One actionable thing we could do is put some exception handling (try catch) in Cesium3DTilesetImpl::Update, link.

It would give us a chance to catch any known types of exceptions and log. This might help distinguish out of memory problems vs other problems.

meta-meta commented 2 days ago

Are there any workarounds you can recommend? In Unity on the Quest, it's inevitable that spending enough time in the app will produce a crash due to running out of memory. We've attempted to hook into Unity's Application.lowMemory event and destroy/recreate the CesiumGeoreference and Cesium3DTileset. That seems to buy some time but in practice, Unity's event happens long before a crash would have happened so it becomes a UX nuisance and the app will still crash.