search

hrydgard / ppsspp

A PSP emulator for Android, Windows, Mac and Linux, written in C++. Want to contribute? Join us on Discord at https://discord.gg/5NJB6dD or just send pull requests / issues. For discussion use the forums at forums.ppsspp.org.
https://www.ppsspp.org
Other
11.41k stars 2.19k forks source link

The 1.14 Google Play mystery crash thread #16601

Closed hrydgard closed 1 year ago

hrydgard commented 1 year ago

More than five years after I first requested it, the Google Play gods have finally answered my prayers and now display assert messages along with stack traces:

image

This means that we can now pass along information about which game was running when we hit an assert, and similar things!

Unfortunately they also made the stack trace UI incredibly slow. Oh well.

There are a whole bunch of OpenGL initialization crashes as usual, now collected better thanks to some asserts I added like that one, but I still think should probably avoid crashing, show a message and bail, or even automatically restart the activity. The few times I've been able to randomly repro these (almost entirely on very old devices), restarting the app helped.

Anyway, if they fix their server issues, I'll add some more mysterious ones here. I did see one in UninstallExceptionHandler but now I can't get it to display it again.

Hm, seems the UninstallExceptionHandler one is just some somewhat worrying memory corruption:

 #00  pc 0x000000000004e178  /apex/com.android.runtime/lib64/bionic/libc.so (abort+168)
  #01  pc 0x0000000000042818  /apex/com.android.runtime/lib64/bionic/libc.so (scudo::die()+8)
  #02  pc 0x0000000000042e50  /apex/com.android.runtime/lib64/bionic/libc.so (scudo::ScopedErrorReport::~ScopedErrorReport()+32)
  #03  pc 0x0000000000042f18  /apex/com.android.runtime/lib64/bionic/libc.so (scudo::reportHeaderCorruption(void*)+60)
  #04  pc 0x00000000000444a0  /apex/com.android.runtime/lib64/bionic/libc.so (scudo::Allocator<scudo::AndroidConfig, &(scudo_malloc_postinit)>::deallocate(void*, scudo::Chunk::Origin, unsigned long, unsigned long)+332)
  #05  pc 0x00000000007646e8  /data/app/~~_MCmDAcVR2sC4CsnTSQpyg==/org.ppsspp.ppsspp-xYIsGg1ImySc_IODjV9WhQ==/split_config.arm64_v8a.apk!libppsspp_jni.so (UninstallExceptionHandler()+116)
  #06  pc 0x00000000005d968c  /data/app/~~_MCmDAcVR2sC4CsnTSQpyg==/org.ppsspp.ppsspp-xYIsGg1ImySc_IODjV9WhQ==/split_config.arm64_v8a.apk!libppsspp_jni.so (CPU_Shutdown()+16)

This surely must be some kind of shutdown bug? Unless there's some curRenderStep_ confusion (== 0 for some reason)...

backtrace:
  #00  pc 0x000000000049587e  _v7a.apk!libppsspp_jni.so (GLRenderManager::SetNoBlendAndMask(int)+106)
  #01  pc 0x0000000000495311  _v7a.apk!libppsspp_jni.so (DrawEngineGLES::ApplyDrawState(int)+288)
  #02  pc 0x0000000000499071  _v7a.apk!libppsspp_jni.so (DrawEngineGLES::DoFlush()+1664)
  #03  pc 0x00000000005087f9  _v7a.apk!libppsspp_jni.so (GPUCommon::FastRunLoop(DisplayList&)+192)
  #04  pc 0x000000000050838f  _v7a.apk!libppsspp_jni.so (GPUCommon::InterpretList(DisplayList&)+410)
  #05  pc 0x0000000000507c4d  _v7a.apk!libppsspp_jni.so (GPUCommon::ProcessDLQueue()+104)
  #06  pc 0x0000000000507b13  _v7a.apk!libppsspp_jni.so (GPUCommon::EnqueueList(unsigned int, unsigned int, int, PSPPointer<PspGeListArgs>, bool)+1462)
  #07  pc 0x00000000003a1519  _v7a.apk!libppsspp_jni.so (void WrapU_UUIU<&(sceGeListEnQueue(unsigned int, unsigned int, int, unsigned int))>()+48)
  #08  pc 0x00000000003842a5  _v7a.apk!libppsspp_jni.so (CallSyscallWithoutFlags(HLEFunction const*)+32)
  #09  pc 0x0000000000002022  [heap]

Locking a destroyed mutex, seems like in a VK driver:

  #00  pc 0x0000000000078b9c  /apex/com.android.runtime/lib64/bionic/libc.so (abort+180)
  #01  pc 0x000000000007a9fc  /apex/com.android.runtime/lib64/bionic/libc.so (__fortify_fatal(char const*, ...)+124)
  #02  pc 0x00000000000dd1cc  /apex/com.android.runtime/lib64/bionic/libc.so (HandleUsingDestroyedMutex(pthread_mutex_t*, char const*)+60)
  #03  pc 0x00000000000dd00c  /apex/com.android.runtime/lib64/bionic/libc.so (pthread_mutex_lock+172)
  #04  pc 0x00000000006cf7e8  /vendor/lib64/egl/libGLES_mali.so
  #05  pc 0x00000000007eba54  /vendor/lib64/egl/libGLES_mali.so
  #06  pc 0x00000000006cd898  /vendor/lib64/egl/libGLES_mali.so
  #07  pc 0x00000000006cc328  /vendor/lib64/egl/libGLES_mali.so
  #08  pc 0x00000000006cb320  /data/app/~~Z48YEGq3eSqmDamiLrUOWg==/org.ppsspp.ppsspp-uprapXiBl8CpmMwHp2grWA==/split_config.arm64_v8a.apk!libppsspp_jni.so (VKRGraphicsPipeline::Create(VulkanContext*)+80)
  #09  pc 0x00000000006cce4c  /data/app/~~Z48YEGq3eSqmDamiLrUOWg==/org.ppsspp.ppsspp-uprapXiBl8CpmMwHp2grWA==/split_config.arm64_v8a.apk!libppsspp_jni.so (VulkanRenderManager::CompileThreadFunc()+256)
  #10  pc 0x00000000006d0e2c  /data/app/~~Z48YEGq3eSqmDamiLrUOWg==/org.ppsspp.ppsspp-uprapXiBl8CpmMwHp2grWA==/split_config.arm64_v8a.apk!libppsspp_jni.so (void* std::__ndk1::__thread_proxy<std::__ndk1::tuple<std::__ndk1::unique_ptr<std::__ndk1::__thread_struct, std::__ndk1::default_delete<std::__ndk1::__thread_struct> >, void (VulkanRenderManager::*)(), VulkanRenderManager*> >(void*)+64)
unknownbrackets commented 1 year ago

Hm, SetNoBlendAndMask would be the first opportunity for ApplyDrawState() to interact with a missing curRenderStep_ in a lot of cases.

(VKRGraphicsPipeline::Create(VulkanContext*)+80) seems relatively early for the Vk calls in that func, but... if this isn't a driver bug it seems like it would either need to be a shutdown hazard or a pipeline cache lifetime issue, right?

-[Unknown]

hrydgard commented 1 year ago

Found a hang reported, where one of the threads were sitting here, very odd:

#00  pc 0x000000000055c5e0  !libppsspp_jni.so (JitBlockCache::GetBlockNumbersFromAddress(unsigned int, std::__ndk1::vector<int, std::__ndk1::allocator<int> >*)+128)
#01  pc 0x00000000003d5604 !libppsspp_jni.so (MIPSComp::Arm64Jit::DescribeCodePtr(unsigned char const*, std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> >&)+176)
#02  pc 0x0000000000574f20 !libppsspp_jni.so (Memory::HandleFault(unsigned long, void*)+300)

Wonder if we're just handling a fault over and over, or if this was just random..

hrydgard commented 1 year ago

Alright, so now after 1.14 there are some updates:

Strange TransformUnit::Flush crash (software renderer):

backtrace:
  #00  pc 0x00000000006c2070  libppsspp_jni.so (TransformUnit::Flush(char const*)+12)
  #01  pc 0x00000000006936d8  libppsspp_jni.so (GPUCommon::InterpretList(DisplayList&)+688)
  #02  pc 0x0000000000692bd0  libppsspp_jni.so (GPUCommon::ProcessDLQueue()+100)
  #03  pc 0x0000000000693068  libppsspp_jni.so (GPUCommon::UpdateStall(int, unsigned int)+88)
  #04  pc 0x00000000004ce294  libppsspp_jni.so
  #05  pc 0x00000000004a7058  libppsspp_jni.so (CallSyscallWithoutFlags(HLEFunction const*)+52)
  #06  pc 0x0000000000007820

Wacky bad_alloc in draw buffer (likely just OOM?):

Thread
terminating with uncaught exception of type std::bad_alloc: std::bad_alloc
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 30603 >>> org.ppsspp.ppsspp <<<

backtrace:
  #00  pc 0x00000000000895ec  /apex/com.android.runtime/lib64/bionic/libc.so (abort+180)
  #01  pc 0x0000000000e22960  libppsspp_jni.so
  #02  pc 0x0000000000e22ab8  libppsspp_jni.so
  #03  pc 0x0000000000e1f9a8  libppsspp_jni.so
  #04  pc 0x0000000000e1efd4  libppsspp_jni.so
  #05  pc 0x0000000000e1ef30  libppsspp_jni.so (__cxa_throw+112)
  #06  pc 0x0000000000e156c0  libppsspp_jni.so (operator new(unsigned long)+96)
  #07  pc 0x0000000000612a70  libppsspp_jni.so (void std::__ndk1::vector<VKRGraphicsPipeline*, std::__ndk1::allocator<VKRGraphicsPipeline*> >::__push_back_slow_path<VKRGraphicsPipeline* const&>(VKRGraphicsPipeline* const&)+120)
  #08  pc 0x0000000000c597a4  libppsspp_jni.so (Draw::VKContext::DrawUP(void const*, int)+452)
  #09  pc 0x0000000000770588  libppsspp_jni.so (DrawBuffer::Flush(bool)+132)
  #10  pc 0x0000000000765aa4  libppsspp_jni.so (UIContext::Flush()+28)
  #11  pc 0x000000000078c464  libppsspp_jni.so (EmuScreen::renderUI()+3404)
  #12  pc 0x000000000078b3b0  libppsspp_jni.so (EmuScreen::render()+216)
  #13  pc 0x0000000000c61050  libppsspp_jni.so (ScreenManager::render()+212)
  #14  pc 0x0000000000781a74  libppsspp_jni.so (NativeRender(GraphicsContext*)+696)
  #15  pc 0x000000000077a5e8  libppsspp_jni.so (Java_org_ppsspp_ppsspp_NativeActivity_runVulkanRenderLoop+428)
hrydgard commented 1 year ago

ReadFromZip:

  #00  pc 0x0000000000c83430  libppsspp_jni.so (_zip_error_set_from_source+32)
  #01  pc 0x0000000000c88014  libppsspp_jni.so (zip_source_open+208)
  #02  pc 0x0000000000c87fb0  libppsspp_jni.so (zip_source_open+108)
  #03  pc 0x0000000000c87fb0  libppsspp_jni.so (zip_source_open+108)
  #04  pc 0x0000000000c87fb0  libppsspp_jni.so (zip_source_open+108)
  #05  pc 0x0000000000c87fb0  libppsspp_jni.so (zip_source_open+108)
  #06  pc 0x0000000000c8440c  libppsspp_jni.so (zip_fopen_index_encrypted+56)
  #07  pc 0x0000000000c4bca4  libppsspp_jni.so (ReadFromZip(zip*, char const*, unsigned long*)+52)
  #08  pc 0x0000000000c4bfbc  libppsspp_jni.so (ZipAssetReader::ReadAsset(char const*, unsigned long*)+92)
  #09  pc 0x00000000007157c8  libppsspp_jni.so (VFSReadFile(char const*, unsigned long*)+324)
  #10  pc 0x00000000007917fc  libppsspp_jni.so
  #11  pc 0x0000000000790680  libppsspp_jni.so (GameInfoWorkItem::Run()+1792)
  #12  pc 0x0000000000763514  libppsspp_jni.so
  #13  pc 0x0000000000764da8  libppsspp_jni.so (void* std::__ndk1::__thread_proxy<std::__ndk1::tuple<std::__ndk1::unique_ptr<std::__ndk1::__thread_struct, std::__ndk1::default_delete<std::__ndk1::__thread_struct>>, void (*)(GlobalThreadContext*, ThreadContext*), GlobalThreadContext*, ThreadContext*>>(void*)+48)
  #14  pc 0x00000000000d4358  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+36)
  #15  pc 0x0000000000071aa8  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64)

Mic input (!)


  #00  pc 0x000000000004d4b4  /apex/com.android.runtime/lib64/bionic/libc.so (__memcpy+276)
  #01  pc 0x0000000000584490  libppsspp_jni.so (QueueBuf::pop(unsigned char*, unsigned int)+136)
  #02  pc 0x0000000000584948  libppsspp_jni.so (__MicInput(unsigned int, unsigned int, unsigned int, MICTYPE, bool)+408)
  #03  pc 0x000000000058217c  libppsspp_jni.so
  #04  pc 0x00000000004a7058  libppsspp_jni.so (CallSyscallWithoutFlags(HLEFunction const*)+52)
  #05  pc 0x000000000004b444

JIT finalize block:

backtrace:
  #00  pc 0x00000000003116b0  libppsspp_jni.so
  #01  pc 0x00000000003114c1  libppsspp_jni.so
  #02  pc 0x00000000003104fb  libppsspp_jni.so (JitBlockCache::FinalizeBlock(int, bool)+194)
  #03  pc 0x0000000000304c09  libppsspp_jni.so (MIPSComp::ArmJit::Compile(unsigned int)+140)
  #04  pc 0x0000000000000106
unknownbrackets commented 1 year ago

Strange TransformUnit::Flush crash (software renderer):

Hm, unless binner_ is null or broken or something, not sure. Memory corruption? OOM when even allocating binner? It is a bit large with all its buffers.

ReadFromZip:

Strange. Not sure if we end up in the "layered" case, but this is a read from assets even? That zip shouldn't be corrupt or anything... it should be a user zip.

Mic input (!)

__MicInput doesn't seem to check the pointer validity at all, but this seems to be to a newd temp buffer. Maybe even also OOM...

Not sure much about the others.

-[Unknown]

hrydgard commented 1 year ago

Been using addr2line on some stack traces that didn't resolve in the UI, there's one in the texture replacer here:

        ParallelRangeLoop(&g_threadManager, [&](int l, int h) {
            for (int y = l; y < h; ++y) {
                memcpy((uint8_t *)out + rowPitch * y, &data[0] + info.w * 4 * y, info.w * 4);  << bad
            }
        }, 0, info.h, MIN_LINES_PER_THREAD);

Also made a bunch of fixes in #16683

hrydgard commented 1 year ago

The most common hang has the following as the only interesting stack trace:

  #00  pc 0x0000000000086b8c  /apex/com.android.runtime/lib64/bionic/libc.so (syscall+28)
  #01  pc 0x000000000008a938  /apex/com.android.runtime/lib64/bionic/libc.so (__futex_wait_ex(void volatile*, bool, int, bool, timespec const*)+144)
  #02  pc 0x00000000000ec784  /apex/com.android.runtime/lib64/bionic/libc.so (NonPI::MutexLockWithTimeout(pthread_mutex_internal_t*, bool, timespec const*)+688)
  #03  pc 0x0000000000e15010  libppsspp_jni.so (std::__ndk1::recursive_mutex::lock()+8)
  #04  pc 0x0000000000c60cf8  libppsspp_jni.so (ScreenManager::axis(AxisInput const&)+60)
  #05  pc 0x0000000000782a60  libppsspp_jni.so (NativeAxis(AxisInput const&)+304)
  #06  pc 0x00000000007790cc  libppsspp_jni.so (Java_org_ppsspp_ppsspp_NativeApp_accelerometer+84)

This is the only other stack that had PPSSPP code in it:

  #00  pc 0x0000000000086b8c  /apex/com.android.runtime/lib64/bionic/libc.so (syscall+28)
  #01  pc 0x000000000008a938  /apex/com.android.runtime/lib64/bionic/libc.so (__futex_wait_ex(void volatile*, bool, int, bool, timespec const*)+144)
  #02  pc 0x00000000000eab90  /apex/com.android.runtime/lib64/bionic/libc.so (pthread_cond_wait+60)
  #03  pc 0x0000000000dda8e0  libppsspp_jni.so (std::__ndk1::condition_variable::wait(std::__ndk1::unique_lock<std::__ndk1::mutex>&)+20)
  #04  pc 0x0000000000763560  libppsspp_jni.so
  #05  pc 0x0000000000764da8  libppsspp_jni.so (void* std::__ndk1::__thread_proxy<std::__ndk1::tuple<std::__ndk1::unique_ptr<std::__ndk1::__thread_struct, std::__ndk1::default_delete<std::__ndk1::__thread_struct> >, void (*)(GlobalThreadContext*, ThreadContext*), GlobalThreadContext*, ThreadContext*> >(void*)+48)

which makes me think it's during shutdown or startup.

hrydgard commented 1 year ago

The second most common hang is a Vulkan shutdown hang, with the following stack traces:

#02  pc 0x0000000000e1d4dc  libppsspp_jni.so (std::__ndk1::thread::join()+28)
#03  pc 0x00000000007442e4  libppsspp_jni.so (VulkanRenderManager::StopThread()+432)
#04  pc 0x00000000007444a8  libppsspp_jni.so (VulkanRenderManager::DestroyBackbuffers()+16)
#05  pc 0x000000000077c8e4  libppsspp_jni.so (AndroidVulkanContext::ShutdownFromRenderThread()+80)
#06  pc 0x000000000077a678  libppsspp_jni.so (Java_org_ppsspp_ppsspp_NativeActivity_runVulkanRenderLoop+572)
#00  pc 0x00000000000c67f8  /apex/com.android.runtime/lib64/bionic/libc.so (nanosleep+8)
#01  pc 0x000000000008b6bc  /apex/com.android.runtime/lib64/bionic/libc.so (usleep+76)
#02  pc 0x0000000000779870  !libppsspp_jni.so (Java_org_ppsspp_ppsspp_NativeActivity_requestExitVulkanRenderLoop+96)
at org.ppsspp.ppsspp.NativeActivity.requestExitVulkanRenderLoop (Native method)
at org.ppsspp.ppsspp.NativeActivity.joinRenderLoopThread (NativeActivity.java:718)
at org.ppsspp.ppsspp.NativeActivity.onPause (NativeActivity.java:805)
at android.app.Activity.performPause (Activity.java:8333)
at android.app.Instrumentation.callActivityOnPause (Instrumentation.java:1510)
hrydgard commented 1 year ago

Another, I think, GL one:

#00  pc 0x000000000009a77c  /apex/com.android.runtime/lib/bionic/libc.so (getuid+12)
#01  pc 0x000000000000a41d  /system/lib/liblog.so (PmsgWrite(log_id, timespec*, iovec*, unsigned int)+320)
#02  pc 0x0000000000004ee9  /system/lib/liblog.so (write_to_log(log_id, iovec*, unsigned int)+128)
#03  pc 0x0000000000004e4f  /system/lib/liblog.so (__android_log_logd_logger+66)
#04  pc 0x000000000000c923  /system/lib/libbase.so (android::base::LogdLogChunk(android::base::LogId, android::base::LogSeverity, char const*, char const*)+98)
#05  pc 0x000000000000c87f  /system/lib/libbase.so (void android::base::SplitByLogdChunks<void (android::base::LogId, android::base::LogSeverity, char const*, char const*)>(android::base::LogId, android::base::LogSeverity, char const*, char const*, unsigned int, char const*, void  const(&)(android::base::LogId, android::base::LogSeverity, char const*, char const*))+834)
#06  pc 0x000000000000c535  /system/lib/libbase.so (android::base::LogdLogger::operator()(android::base::LogId, android::base::LogSeverity, char const*, char const*, unsigned int, char const*)+30)
#07  pc 0x000000000001a24d  /system/lib/libartbase.so (std::__1::__function::__func<art::InitLogging(char**, void (&)(char const*))::LogdLoggerLocked, std::__1::allocator<art::InitLogging(char**, void (&)(char const*))::LogdLoggerLocked>, void (android::base::LogId, android::base::LogSeverity, char const*, char const*, unsigned int, char const*)>::operator()(android::base::LogId&&, android::base::LogSeverity&&, char const*&&, char const*&&, unsigned int&&, char const*&&)+76)
#08  pc 0x000000000000d8a5  /system/lib/libbase.so (android::base::SetLogger(std::__1::function<void (android::base::LogId, android::base::LogSeverity, char const*, char const*, unsigned int, char const*)>&&)::$_2::__invoke(__android_log_message const*)+120)
#09  pc 0x00000000000050c9  /system/lib/liblog.so (__android_log_write_log_message+88)
#10  pc 0x0000000000005281  /system/lib/liblog.so (__android_log_print+112)
#11  pc 0x00000000005b6c73  !libppsspp_jni.so (AndroidLogger::Log(LogMessage const&)+82)
#12  pc 0x00000000005ad797  !libppsspp_jni.so (LogManager::Log(LogTypes::LOG_LEVELS, LogTypes::LOG_TYPE, char const*, int, char const*, std::__va_list)+370)
#13  pc 0x00000000005ad5d9  !libppsspp_jni.so (GenericLog(LogTypes::LOG_LEVELS, LogTypes::LOG_TYPE, char const*, int, char const*, ...)+68)
#14  pc 0x0000000000ade36d  !libppsspp_jni.so (ScreenManager::render()+64)
#15  pc 0x00000000005c1373  !libppsspp_jni.so (NativeRender(GraphicsContext*)+558)
#16  pc 0x00000000005ba157  !libppsspp_jni.so (UpdateRunLoopAndroid(_JNIEnv*)+26)
#17  pc 0x00000000005bbcd1  !libppsspp_jni.so
#18  pc 0x000000000032b88d  !libppsspp_jni.so (void* std::__ndk1::__thread_proxy<std::__ndk1::tuple<std::__ndk1::unique_ptr<std::__ndk1::__thread_struct, std::__ndk1::default_delete<std::__ndk1::__thread_struct> >, void (*)()> >(void*)+24)
#19  pc 0x00000000000aad13  /apex/com.android.runtime/lib/bionic/libc.so (__pthread_start(void*)+40)
#20  pc 0x0000000000064193  /apex/com.android.runtime/lib/bionic/libc.so (__start_thread+30)

#00  pc 0x000000000005e3e8  /apex/com.android.runtime/lib/bionic/libc.so (syscall+28)
#01  pc 0x0000000000063559  /apex/com.android.runtime/lib/bionic/libc.so (__futex_wait_ex(void volatile*, bool, int, bool, timespec const*)+92)
#02  pc 0x00000000000aa393  /apex/com.android.runtime/lib/bionic/libc.so (pthread_cond_wait+32)
#03  pc 0x0000000000bfbe49  !libppsspp_jni.so (std::__ndk1::condition_variable::wait(std::__ndk1::unique_lock<std::__ndk1::mutex>&)+12)
#04  pc 0x000000000057def7  !libppsspp_jni.so (GLRenderManager::ThreadFrame()+142)
#05  pc 0x00000000005ba235  !libppsspp_jni.so (Java_org_ppsspp_ppsspp_NativeRenderer_displayRender+64)
#06  pc 0x0000000000009053  /oat/arm/base.odex (Java_org_ppsspp_ppsspp_NativeRenderer_displayRender__+74)
at org.ppsspp.ppsspp.NativeRenderer.displayRender (Native method)
at org.ppsspp.ppsspp.NativeRenderer.onDrawFrame (NativeRenderer.java:33)
at android.opengl.GLSurfaceView$GLThread.guardedRun (GLSurfaceView.java:1591)
at android.opengl.GLSurfaceView$GLThread.run (GLSurfaceView.java:1286)
at java.lang.Object.wait (Native method)
  at java.lang.Object.wait (Object.java:442)
  at java.lang.Object.wait (Object.java:568)
  at android.opengl.GLSurfaceView$GLThread.guardedRun (GLSurfaceView.java:1515)
  at android.opengl.GLSurfaceView$GLThread.run (GLSurfaceView.java:1286)
hrydgard commented 1 year ago

Also unrelated, another TransformUnit one, but surely also OOM:

  #00  pc 0x000000000052c05c  !libppsspp_jni.so (TransformUnit::SetDirty(SoftDirty))
  #01  pc 0x00000000005272ed  !libppsspp_jni.so (SoftGPU::Execute_Prim(unsigned int, unsigned int)+296)
  #02  pc 0x0000000000529549  !libppsspp_jni.so (SoftGPU::FastRunLoop(DisplayList&)+72)
unknownbrackets commented 1 year ago

Another, I think, GL one:

That's weird, ScreenManager::render() only logs in bad situations - transparent over nothing or no screen. That probably would just hang...

-[Unknown]

hrydgard commented 1 year ago

That is indeed a reported hang (ANR - application not responding), not a crash. No idea how it got in that situation..

hrydgard commented 1 year ago

Here's an oldie but goodie that have never fully gone away:

  #00  pc 0x00000000006951b4  !libppsspp_jni.so (GPUCommon::DoExecuteCall(unsigned int)+52)
  #01  pc 0x0000000000693b4c  !libppsspp_jni.so (GPUCommon::FastRunLoop(DisplayList&)+92)
  #02  pc 0x000000000069367c  !libppsspp_jni.so (GPUCommon::InterpretList(DisplayList&)+596)
  #03  pc 0x0000000000692bd0  !libppsspp_jni.so (GPUCommon::ProcessDLQueue()+100)
  #04  pc 0x0000000000693068  !libppsspp_jni.so (GPUCommon::UpdateStall(int, unsigned int)+88)
  #05  pc 0x00000000004ce294  !libppsspp_jni.so
  #06  pc 0x00000000004a7058  !libppsspp_jni.so (CallSyscallWithoutFlags(HLEFunction const*)+52)
  #07  pc 0x0000000000010eac

I'm considering throwing in an _assertmsg so we can figure out which game it is happening in.. But surely it's from executing a bogus display list somehow.

hrydgard commented 1 year ago

argh, looks like I caused a new kind of java exception:

Exception java.lang.RuntimeException: Can't toast on a thread that has not called Looper.prepare()
  at android.widget.Toast$TN.<init> (Toast.java:411)
  at android.widget.Toast.<init> (Toast.java:121)
  at android.widget.Toast.makeText (Toast.java:290)
  at android.widget.Toast.makeText (Toast.java:280)
  at org.ppsspp.ppsspp.NativeRenderer.onSurfaceCreated (NativeRenderer.java:60)
  at android.opengl.GLSurfaceView$GLThread.guardedRun (GLSurfaceView.java:1541)
  at android.opengl.GLSurfaceView$GLThread.run (GLSurfaceView.java:1272)

trying to display an error message:

https://github.com/hrydgard/ppsspp/blob/bd2a0c99bb01ab423ab93d07a1724c035ca48460/android/src/org/ppsspp/ppsspp/NativeRenderer.java#L60

Oh well, at that point we're already screwed..

hrydgard commented 1 year ago

Also hit this one:

(GLRenderManager.h:SetBlendAndMask:801) Critical: [curRenderStep && curRenderStep->stepType == GLRStepType::RENDER] Assertion

Seems the thing I added to include the game ID isn't working? darn. don't see how that's possible though? Maybe we hit a length limit?

hrydgard commented 1 year ago

Got a few variants of these: art/runtime/thread.cc:1238] Native thread exited without calling DetachCurrentThread: Thread[14,tid=30521,Native,Thread*=0x9ba87900,peer=0x12dc60a0,"Thread-1467"]

I don't think Android normally asserts in this situation, might be a weird OS build. But, we are not handling thread detachment properly in all cases - getEnv() just attaches, and there's no mechanism to detach.

This one we can't do much about, though we can handle it more gracefully:

(MemArenaAndroid.cpp:Find4GBBase:160) Critical: [base != MAP_FAILED] Failed to map 256 MB of memory space: Out of memory (ULES00502 Grand Theft Auto: Vice City Stories)

A bit weird (32-bit):

  #00  pc 0x0000000000691a74  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (XXH3_hashLong_64b_default)
  #01  pc 0x00000000006916c9  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (XXH3_64bits+596)
  #02  pc 0x00000000003107db  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (JitBlockCache::FinalizeBlock(int, bool)+98)
  #03  pc 0x0000000000304f49  /data/app/org.ppsspp.ppsspp-1/lib/arm/libppsspp_jni.so (MIPSComp::ArmJit::Compile(unsigned int)+140)
ghost commented 1 year ago

This can be considered as an issue? Screenshot_20230103_174655

I downloaded apk from ppsspp.org and I try to update it on playstore.

hrydgard commented 1 year ago

That's expected, you need to uninstall to switch between APK and playstore due to mismatching signing keys.

I currently don't know of a way to fix that.

hrydgard commented 1 year ago

Here's one I haven't noticed before (as we stomp these out, I discover rarer crashes in the pile):

Thread
FORTIFY: pthread_mutex_destroy called on a destroyed mutex (0xd23f5370)
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 23399 >>> org.ppsspp.ppsspp <<<

backtrace:
  #00  pc 0x0000000000062d00  /apex/com.android.runtime/lib/bionic/libc.so (abort+172)
  #01  pc 0x00000000000ac23f  /apex/com.android.runtime/lib/bionic/libc.so (__fortify_fatal(char const*, ...)+26)
  #02  pc 0x00000000000ab979  /apex/com.android.runtime/lib/bionic/libc.so (HandleUsingDestroyedMutex(pthread_mutex_t*, char const*)+20)
  #03  pc 0x00000000000ac11f  /apex/com.android.runtime/lib/bionic/libc.so (pthread_mutex_destroy+126)
  #04  pc 0x0000000000c18971  libppsspp_jni.so (std::__ndk1::mutex::~mutex()+8)
  #05  pc 0x0000000000446433  libppsspp_jni.so (LocalFileLoader::~LocalFileLoader()+30)
  #06  pc 0x0000000000446469  libppsspp_jni.so (LocalFileLoader::~LocalFileLoader()+4)
  #07  pc 0x00000000005ce427  libppsspp_jni.so (GameInfoWorkItem::Run()+402)
  #08  pc 0x00000000005aaf11  libppsspp_jni.so
  #09  pc 0x00000000005ac0d9  libppsspp_jni.so (void* std::__ndk1::__thread_proxy<std::__ndk1::tuple<std::__ndk1::unique_ptr<std::__ndk1::__thread_struct, std::__ndk1::default_delete<std::__ndk1::__thread_struct> >, void (*)(GlobalThreadContext*, ThreadContext*), GlobalThreadContext*, ThreadContext*> >(void*)+28)
  #10  pc 0x00000000000aaf93  /apex/com.android.runtime/lib/bionic/libc.so (__pthread_start(void*)+40)
  #11  pc 0x0000000000064203  /apex/com.android.runtime/lib/bionic/libc.so (__start_thread+30)
hrydgard commented 1 year ago

This should just be an OOM, right?

Thread
/buildbot/src/android/ndk-release-r21/external/libcxx/../../external/libcxxabi/src/abort_message.cpp:72: abort_message: assertion "terminating with uncaught exception of type std::bad_alloc: std::bad_alloc" failed
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 13497 >>> org.ppsspp.ppsspp <<<

backtrace:
  #00  pc 0x000000000001ccb6  /system/lib/libc.so (abort+58)
  #01  pc 0x000000000001cf2f  /system/lib/libc.so (__assert2+22)
  #02  pc 0x0000000000c1fe3d  armeabi_v7a.apk
  #03  pc 0x0000000000c1ff3d  armeabi_v7a.apk
  #04  pc 0x0000000000c1e4fd  armeabi_v7a.apk
  #05  pc 0x0000000000c1ddab  armeabi_v7a.apk
  #06  pc 0x0000000000c1dd73  armeabi_v7a.apk (__cxa_throw+74)
  #07  pc 0x0000000000c189dd  armeabi_v7a.apk (operator new(unsigned int)+56)
  #08  pc 0x000000000034482f  armeabi_v7a.apk (std::__ndk1::vector<unsigned char, std::__ndk1::allocator<unsigned char>>::__append(unsigned int)+100)
  #09  pc 0x00000000004665a3  armeabi_v7a.apk (SaveState::SaveToRam(std::__ndk1::vector<unsigned char, std::__ndk1::allocator<unsigned char>>&)+118)
  #10  pc 0x000000000046ba11  armeabi_v7a.apk (SaveState::StateRingbuffer::Save()+132)
  #11  pc 0x000000000046af03  armeabi_v7a.apk (SaveState::Process()+3314)
  #12  pc 0x000000000046ee77  armeabi_v7a.apk (PSP_RunLoopWhileState()+94)
  #13  pc 0x00000000005ca65d  armeabi_v7a.apk (EmuScreen::render()+220)
  #14  pc 0x0000000000adf29b  armeabi_v7a.apk (ScreenManager::render()+126)
  #15  pc 0x00000000005c20b7  armeabi_v7a.apk (NativeRender(GraphicsContext*)+558)
  #16  pc 0x00000000005bc2c7  armeabi_v7a.apk (Java_org_ppsspp_ppsspp_NativeActivity_runVulkanRenderLoop+334)
  #17  pc 0x0000000000009105  /data/app/org.ppsspp.ppsspp-ARrsgkSFQAgRLhxXT4yFDQ==/oat/arm/base.odex (org.ppsspp.ppsspp.NativeActivity.runVulkanRenderLoop+92)
hrydgard commented 1 year ago 
backtrace:
  #00  pc 0x00000000003105c2  /data/app/org.ppsspp.ppsspp-ULG9IAVcsI4f8P_kWyxoxw==/split_config.armeabi_v7a.apk (JitBlockCache::ProxyBlock(unsigned int, unsigned int, unsigned int, unsigned char const*)+278)
  #01  pc 0x000000000030599f  /data/app/org.ppsspp.ppsspp-ULG9IAVcsI4f8P_kWyxoxw==/split_config.armeabi_v7a.apk (MIPSComp::ArmJit::ReplaceJalTo(unsigned int)+510)
  #02  pc 0x00000000002f8d79  /data/app/org.ppsspp.ppsspp-ULG9IAVcsI4f8P_kWyxoxw==/split_config.armeabi_v7a.apk (MIPSComp::ArmJit::Comp_Jump(Memory::Opcode)+204)
  #03  pc 0x0000000000456dcb  /data/app/org.ppsspp.ppsspp-ULG9IAVcsI4f8P_kWyxoxw==/split_config.armeabi_v7a.apk (MIPSCompileOp(Memory::Opcode, MIPSComp::MIPSFrontendInterface*)+166)
  #04  pc 0x0000000000305229  /data/app/org.ppsspp.ppsspp-ULG9IAVcsI4f8P_kWyxoxw==/split_config.armeabi_v7a.apk (MIPSComp::ArmJit::DoJit(unsigned int, JitBlock*)+424)
  #05  pc 0x0000000000304f3d  /data/app/org.ppsspp.ppsspp-ULG9IAVcsI4f8P_kWyxoxw==/split_config.armeabi_v7a.apk (MIPSComp::ArmJit::Compile(unsigned int)+128)
  #06  pc 0x0000000000000106

This one is actually interesting. I think we might be missing an IsFull and wipe check before we link/proxy blocks...

hrydgard commented 1 year ago 
Thread
/buildbot/src/android/ndk-release-r21/external/libcxx/../../external/libcxxabi/src/abort_message.cpp:72: abort_message: assertion "terminating" failed
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 26034 >>> org.ppsspp.ppsspp <<<

backtrace:
  #00  pc 0x0000000000065668  /apex/com.android.runtime/lib/bionic/libc.so (abort+172)
  #01  pc 0x00000000000658d7  /apex/com.android.runtime/lib/bionic/libc.so (__assert2+22)
  #02  pc 0x0000000000c1fe3d  libppsspp_jni.so
  #03  pc 0x0000000000c1fec3  libppsspp_jni.so
  #04  pc 0x0000000000c1e4fd  libppsspp_jni.so
  #05  pc 0x0000000000c1e495  libppsspp_jni.so (std::terminate()+28)
  #06  pc 0x0000000000c1cc1d  libppsspp_jni.so (std::__ndk1::thread::~thread()+12)
  #07  pc 0x000000000047b459  libppsspp_jni.so (std::__ndk1::__shared_ptr_pointer<std::__ndk1::thread*, std::__ndk1::default_delete<std::__ndk1::thread>, std::__ndk1::allocator<std::__ndk1::thread> >::__on_zero_shared()+12)
  #08  pc 0x0000000000477f87  libppsspp_jni.so (GameManager::InstallGameOnThread(Path const&, Path const&, bool)+154)
  #09  pc 0x0000000000477e65  libppsspp_jni.so (GameManager::Update()+624)
  #10  pc 0x00000000005c1ead  libppsspp_jni.so (NativeRender(GraphicsContext*)+36)
  #11  pc 0x00000000005bae57  libppsspp_jni.so (UpdateRunLoopAndroid(_JNIEnv*)+26)
  #12  pc 0x00000000005bc9e1  libppsspp_jni.so
  #13  pc 0x000000000032bbcd  libppsspp_jni.so (void* std::__ndk1::__thread_proxy<std::__ndk1::tuple<std::__ndk1::unique_ptr<std::__ndk1::__thread_struct, std::__ndk1::default_delete<std::__ndk1::__thread_struct> >, void (*)()> >(void*)+24)
  #14  pc 0x00000000000b0567  /apex/com.android.runtime/lib/bionic/libc.so (__pthread_start(void*)+40)
  #15  pc 0x0000000000066b37  /apex/com.android.runtime/lib/bionic/libc.so (__start_thread+30)
hrydgard commented 1 year ago

here's one curious assert that we probably should try to avoid crashing on (omitting the callstack because uninteresting):

(VulkanFrameData.cpp:AcquireNextImage:80): [false] (ULES00502 Grand Theft Auto: Vice City Stories) vkAcquireNextImageKHR failed! result=VK_TIMEOUT
unknownbrackets commented 1 year ago

Are all the most common crashes captured here / potentially resolved? Should we close this for next time around?

-[Unknown]

hrydgard commented 1 year ago

Except for the OOMs which are hard to do much about, enough of it is taken care of for this time around, yeah.