search

pytorch / executorch

On-device AI across mobile, embedded and edge for PyTorch
https://pytorch.org/executorch/
Other
1.73k stars 295 forks source link

v0.4.0-rc2 Android Demo App for Llama crashing on Pixel 8 when first prompt submitted #5582

Open grisaitis opened 23 hours ago

grisaitis commented 23 hours ago

🐛 Describe the bug

I find that when I submit my first prompt on the demo app, the app crashes.

See this screenshot video:

https://github.com/user-attachments/assets/ae445900-fe61-4729-a462-cf8147b0351a

This is the logcat output:

---------------------------- PROCESS STARTED (29659) for package com.example.executorchllamademo ----------------------------
2024-09-24 17:46:45.765 29659-29659 ziparchive              com.example.executorchllamademo      W  Unable to open '/data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/base.dm': No such file or directory
2024-09-24 17:46:45.765 29659-29659 ziparchive              com.example.executorchllamademo      W  Unable to open '/data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/base.dm': No such file or directory
2024-09-24 17:46:45.927 29659-29659 nativeloader            com.example.executorchllamademo      D  Configuring clns-6 for other apk /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/base.apk. target_sdk_version=33, uses_libraries=, library_path=/data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64:/data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/base.apk!/lib/arm64-v8a, permitted_path=/data:/mnt/expand:/data/user/0/com.example.executorchllamademo
2024-09-24 17:46:45.938 29659-29659 GraphicsEnvironment     com.example.executorchllamademo      V  Currently set values for:
2024-09-24 17:46:45.938 29659-29659 GraphicsEnvironment     com.example.executorchllamademo      V    angle_gl_driver_selection_pkgs=[com.android.angle, com.linecorp.b612.android, com.campmobile.snow, com.google.android.apps.tachyon]
2024-09-24 17:46:45.938 29659-29659 GraphicsEnvironment     com.example.executorchllamademo      V    angle_gl_driver_selection_values=[angle, native, native, native]
2024-09-24 17:46:45.938 29659-29659 GraphicsEnvironment     com.example.executorchllamademo      V  com.example.executorchllamademo is not listed in per-application setting
2024-09-24 17:46:45.938 29659-29659 GraphicsEnvironment     com.example.executorchllamademo      V  Neither updatable production driver nor prerelease driver is supported.
2024-09-24 17:46:45.959 29659-29659 Compatibil...geReporter com.example.executorchllamademo      D  Compat change id reported: 279646685; UID 10448; state: DISABLED
2024-09-24 17:46:46.018 29659-29709 vulkan                  com.example.executorchllamademo      D  searching for layers in '/data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64'
2024-09-24 17:46:46.018 29659-29709 vulkan                  com.example.executorchllamademo      D  searching for layers in '/data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/base.apk!/lib/arm64-v8a'
2024-09-24 17:46:46.037 29659-29659 AppCompatDelegate       com.example.executorchllamademo      D  Checking for metadata for AppLocalesMetadataHolderService : Service not found
2024-09-24 17:46:46.075 29659-29659 utorchllamademo         com.example.executorchllamademo      W  Accessing hidden method Landroid/view/View;->computeFitSystemWindows(Landroid/graphics/Rect;Landroid/graphics/Rect;)Z (unsupported, reflection, allowed)
2024-09-24 17:46:46.075 29659-29659 utorchllamademo         com.example.executorchllamademo      W  Accessing hidden method Landroid/view/ViewGroup;->makeOptionalFitsSystemWindows()V (unsupported, reflection, allowed)
2024-09-24 17:46:46.079 29659-29659 Compatibil...geReporter com.example.executorchllamademo      D  Compat change id reported: 210923482; UID 10448; state: ENABLED
2024-09-24 17:46:46.079 29659-29659 Compatibil...geReporter com.example.executorchllamademo      D  Compat change id reported: 63938206; UID 10448; state: DISABLED
2024-09-24 17:46:46.093 29659-29659 Compatibil...geReporter com.example.executorchllamademo      D  Compat change id reported: 171228096; UID 10448; state: ENABLED
2024-09-24 17:46:46.124 29659-29659 Compatibil...geReporter com.example.executorchllamademo      D  Compat change id reported: 303326708; UID 10448; state: DISABLED
2024-09-24 17:46:46.158 29659-29659 ETLogging               com.example.executorchllamademo      D  To get started, select your desired model and tokenizer from the top right corner
2024-09-24 17:46:46.161 29659-29659 Compatibil...geReporter com.example.executorchllamademo      D  Compat change id reported: 237531167; UID 10448; state: DISABLED
2024-09-24 17:46:46.337 29659-29664 utorchllamademo         com.example.executorchllamademo      I  Compiler allocated 4206KB to compile void android.widget.TextView.<init>(android.content.Context, android.util.AttributeSet, int, int)
2024-09-24 17:46:46.640 29659-29664 utorchllamademo         com.example.executorchllamademo      I  Compiler allocated 4666KB to compile void android.view.ViewRootImpl.performTraversals()
2024-09-24 17:46:51.929 29659-29799 ProfileInstaller        com.example.executorchllamademo      D  Installing profile for com.example.executorchllamademo
2024-09-24 17:47:06.899 29659-29665 utorchllamademo         com.example.executorchllamademo      I  Background concurrent mark compact GC freed 3046KB AllocSpace bytes, 2(40KB) LOS objects, 86% free, 3841KB/27MB, paused 323us,5.200ms total 59.694ms
2024-09-24 17:47:25.358 29659-29659 ETLogging               com.example.executorchllamademo      D  mModelType from settings LLAMA_3
2024-09-24 17:47:25.903 29659-29659 VRI[MainActivity]       com.example.executorchllamademo      D  visibilityChanged oldVisibility=true newVisibility=false
2024-09-24 17:47:25.916 29659-29709 HWUI                    com.example.executorchllamademo      D  endAllActiveAnimators on 0xb40000762225bdd0 (RippleDrawable) with handle 0xb4000074b22503e0
2024-09-24 17:47:28.139 29659-29659 VRI[SettingsActivity]   com.example.executorchllamademo      D  updatePointerIcon called with position out of bounds
2024-09-24 17:47:28.159 29659-29659 VRI[SettingsActivity]   com.example.executorchllamademo      D  updatePointerIcon called with position out of bounds
2024-09-24 17:47:28.176 29659-29659 VRI[SettingsActivity]   com.example.executorchllamademo      D  updatePointerIcon called with position out of bounds
2024-09-24 17:47:28.193 29659-29659 VRI[SettingsActivity]   com.example.executorchllamademo      D  updatePointerIcon called with position out of bounds
2024-09-24 17:47:28.210 29659-29659 VRI[SettingsActivity]   com.example.executorchllamademo      D  updatePointerIcon called with position out of bounds
2024-09-24 17:47:28.226 29659-29659 VRI[SettingsActivity]   com.example.executorchllamademo      D  updatePointerIcon called with position out of bounds
2024-09-24 17:47:28.931 29659-29659 WindowOnBackDispatcher  com.example.executorchllamademo      W  sendCancelIfRunning: isInProgress=falsecallback=android.view.ViewRootImpl$$ExternalSyntheticLambda11@efba2ce
2024-09-24 17:47:28.933 29659-29709 HWUI                    com.example.executorchllamademo      D  endAllActiveAnimators on 0xb40000762228c8c0 (RippleDrawable) with handle 0xb4000074b2241980
2024-09-24 17:47:28.980 29659-30259 ETLogging               com.example.executorchllamademo      D  Loading model /data/local/tmp/llama/llama_2_7B_chat.pte with tokenizer /data/local/tmp/llama/tokenizer_llama_2_7b_chat.bin
2024-09-24 17:47:28.999 29659-30259 libc                    com.example.executorchllamademo      W  Access denied finding property "ro.mediatek.platform"
2024-09-24 17:47:28.999 29659-30259 libc                    com.example.executorchllamademo      W  Access denied finding property "ro.chipname"
2024-09-24 17:47:28.999 29659-30259 libc                    com.example.executorchllamademo      W  Access denied finding property "ro.hardware.chipname"
2024-09-24 17:47:29.003 29659-30259 nativeloader            com.example.executorchllamademo      D  Load /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64/libexecutorch.so using ns clns-6 from class loader (caller=/data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/base.apk!classes4.dex): ok
2024-09-24 17:47:29.026 29659-30259 nativeloader            com.example.executorchllamademo      D  Load /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64/libfbjni.so using ns clns-6 from class loader (caller=/data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/base.apk!classes4.dex): ok
2024-09-24 17:47:29.532 29659-29659 VRI[SettingsActivity]   com.example.executorchllamademo      D  visibilityChanged oldVisibility=true newVisibility=false
2024-09-24 17:47:29.549 29659-29659 WindowOnBackDispatcher  com.example.executorchllamademo      W  sendCancelIfRunning: isInProgress=falsecallback=android.view.ViewRootImpl$$ExternalSyntheticLambda11@7f26428
2024-09-24 17:47:52.387 29659-30259 ETLogging               com.example.executorchllamademo      D  Load complete. Model path: /data/local/tmp/llama/llama_2_7B_chat.pte
                                                                                                    Tokenizer path: /data/local/tmp/llama/tokenizer_llama_2_7b_chat.bin
                                                                                                    Temperature: 0.1
                                                                                                    Model loaded time: 23405 ms
2024-09-24 17:47:52.421 29659-29659 utorchllamademo         com.example.executorchllamademo      I  Waiting for a blocking GC NativeAlloc
2024-09-24 17:47:52.495 29659-29665 utorchllamademo         com.example.executorchllamademo      W  Cleared Reference was only reachable from finalizer (only reported once)
2024-09-24 17:47:52.524 29659-29665 utorchllamademo         com.example.executorchllamademo      I  NativeAlloc concurrent mark compact GC freed 1308KB AllocSpace bytes, 7(268KB) LOS objects, 85% free, 4235KB/28MB, paused 354us,3.715ms total 103.740ms
2024-09-24 17:47:52.524 29659-29659 utorchllamademo         com.example.executorchllamademo      I  WaitForGcToComplete blocked NativeAlloc on NativeAlloc for 102.967ms
2024-09-24 17:47:55.174 29659-29659 Compatibil...geReporter com.example.executorchllamademo      D  Compat change id reported: 163400105; UID 10448; state: ENABLED
2024-09-24 17:47:55.176 29659-29659 ImeTracker              com.example.executorchllamademo      I  com.example.executorchllamademo:980d09ca: onRequestShow at ORIGIN_CLIENT reason SHOW_SOFT_INPUT fromUser true
2024-09-24 17:47:55.180 29659-29659 InputMethodManager      com.example.executorchllamademo      D  showSoftInput() view=androidx.appcompat.widget.AppCompatEditText{3efbcf2 VFED..CL. .F.P..ID 115,0-965,92 #7f0800b8 app:id/editTextMessage aid=1073741824} flags=0 reason=SHOW_SOFT_INPUT
2024-09-24 17:47:55.186 29659-29659 Choreographer           com.example.executorchllamademo      I  Skipped 80 frames!  The application may be doing too much work on its main thread.
2024-09-24 17:47:55.201 29659-29659 AssistStructure         com.example.executorchllamademo      I  Flattened final assist data: 3088 bytes, containing 1 windows, 17 views
2024-09-24 17:47:55.208 29659-29670 HWUI                    com.example.executorchllamademo      I  Davey! duration=1344ms; Flags=0, FrameTimelineVsyncId=151756, IntendedVsync=912293811709, Vsync=913625807069, InputEventId=-245770153, HandleInputStart=913628434812, AnimationStart=913628986529, PerformTraversalsStart=913628988156, DrawStart=913629254758, FrameDeadline=912310411709, FrameInterval=913627812375, FrameStartTime=16649942, SyncQueued=913632489784, SyncStart=913632581703, IssueDrawCommandsStart=913632703895, SwapBuffers=913636981198, FrameCompleted=913638822995, DequeueBufferDuration=25187, QueueBufferDuration=252319, GpuCompleted=913638822995, SwapBuffersCompleted=913637359209, DisplayPresentTime=881024008323, CommandSubmissionCompleted=913636981198, 
2024-09-24 17:47:55.398 29659-29659 InsetsController        com.example.executorchllamademo      D  show(ime(), fromIme=true)
2024-09-24 17:47:55.405 29659-30637 InteractionJankMonitor  com.example.executorchllamademo      W  Initializing without READ_DEVICE_CONFIG permission. enabled=false, interval=1, missedFrameThreshold=3, frameTimeThreshold=64, package=com.example.executorchllamademo
2024-09-24 17:47:55.637 29659-29659 ImeTracker              com.example.executorchllamademo      I  com.example.executorchllamademo:980d09ca: onShown
2024-09-24 17:48:00.109 29659-30900 ETLogging               com.example.executorchllamademo      D  starting runnable generate()
2024-09-24 17:48:00.109 29659-30900 ETLogging               com.example.executorchllamademo      D  Running inference.. prompt=<|begin_of_text|><|start_header_id|>system<|end_header_id|>
                                                                                                    thing<|eot_id|><|start_header_id|>user<|end_header_id|>
                                                                                                    hello<|eot_id|><|start_header_id|>assistant<|end_header_id|>
                                                                                                    <|eot_id|><|start_header_id|>user<|end_header_id|>
                                                                                                    hello<|eot_id|>
                                                                                                    <|start_header_id|>assistant<|end_uheader_id|>
2024-09-24 17:48:00.183 29659-30900 libc                    com.example.executorchllamademo      A  Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 30900 (pool-3-thread-1), pid 29659 (utorchllamademo)
---------------------------- PROCESS STARTED (30915) for package com.example.executorchllamademo ----------------------------
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A  Cmdline: com.example.executorchllamademo
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A  pid: 29659, tid: 30900, name: pool-3-thread-1  >>> com.example.executorchllamademo <<<
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A        #01 pc 0000000001b23e94  /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64/libexecutorch.so (et_pal_abort+8) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A        #02 pc 0000000001b23ddc  /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64/libexecutorch.so (executorch::runtime::runtime_abort()+8) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A        #03 pc 0000000001aa15d0  /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64/libexecutorch.so (example::Runner::generate(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&, int, std::__ndk1::function<void (std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&)>, std::__ndk1::function<void (executorch::extension::llm::Stats const&)>, bool)+2888) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A        #04 pc 0000000000189798  /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64/libexecutorch.so (executorch_jni::ExecuTorchLlamaJni::generate(facebook::jni::alias_ref<_jintArray*>, int, int, int, facebook::jni::alias_ref<_jstring*>, int, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>, unsigned char)+376) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A        #05 pc 0000000000189f1c  /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64/libexecutorch.so (facebook::jni::detail::MethodWrapper<int (executorch_jni::ExecuTorchLlamaJni::*)(facebook::jni::alias_ref<_jintArray*>, int, int, int, facebook::jni::alias_ref<_jstring*>, int, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>, unsigned char), &(executorch_jni::ExecuTorchLlamaJni::generate(facebook::jni::alias_ref<_jintArray*>, int, int, int, facebook::jni::alias_ref<_jstring*>, int, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>, unsigned char)), executorch_jni::ExecuTorchLlamaJni, int, facebook::jni::alias_ref<_jintArray*>, int, int, int, facebook::jni::alias_ref<_jstring*>, int, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>, unsigned char>::dispatch(facebook::jni::alias_ref<facebook::jni::detail::JTypeFor<facebook::jni::HybridClass<executorch_jni::ExecuTorchLlamaJni, facebook::jni::detail::BaseHybridClass>::JavaPart, facebook::jni::JObject, void>::_javaobject*>, facebook::jni::alias_ref<_jintArray*>&&, int&&, int&&, int&&, facebook::jni::alias_ref<_jstring*>&&, int&&, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>&&, unsigned char&&)+156) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A        #06 pc 0000000000189dfc  /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64/libexecutorch.so (facebook::jni::detail::FunctionWrapper<int (*)(facebook::jni::alias_ref<facebook::jni::detail::JTypeFor<facebook::jni::HybridClass<executorch_jni::ExecuTorchLlamaJni, facebook::jni::detail::BaseHybridClass>::JavaPart, facebook::jni::JObject, void>::_javaobject*>, facebook::jni::alias_ref<_jintArray*>&&, int&&, int&&, int&&, facebook::jni::alias_ref<_jstring*>&&, int&&, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>&&, unsigned char&&), facebook::jni::detail::JTypeFor<facebook::jni::HybridClass<executorch_jni::ExecuTorchLlamaJni, facebook::jni::detail::BaseHybridClass>::JavaPart, facebook::jni::JObject, void>::_javaobject*, int, facebook::jni::alias_ref<_jintArray*>, int, int, int, facebook::jni::alias_ref<_jstring*>, int, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>, unsigned char>::call(_JNIEnv*, _jobject*, _jintArray*, int, int, int, _jstring*, int, facebook::jni::detail::JTypeFor<executorch_jni::ExecuTorchLlamaCallbackJni, facebook::jni::JObject, void>::_javaobject*, unsigned char, int (*)(facebook::jni::alias_ref<facebook::jni::detail::JTypeFor<facebook::jni::HybridClass<executorch_jni::ExecuTorchLlamaJni, facebook::jni::detail::BaseHybridClass>::JavaPart, facebook::jni::JObject, void>::_javaobject*>, facebook::jni::alias_ref<_jintArray*>&&, int&&, int&&, int&&, facebook::jni::alias_ref<_jstring*>&&, int&&, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>&&, unsigned char&&))+168) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A        #07 pc 00000000001886a4  /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/lib/arm64/libexecutorch.so (facebook::jni::detail::MethodWrapper<int (executorch_jni::ExecuTorchLlamaJni::*)(facebook::jni::alias_ref<_jintArray*>, int, int, int, facebook::jni::alias_ref<_jstring*>, int, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>, unsigned char), &(executorch_jni::ExecuTorchLlamaJni::generate(facebook::jni::alias_ref<_jintArray*>, int, int, int, facebook::jni::alias_ref<_jstring*>, int, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>, unsigned char)), executorch_jni::ExecuTorchLlamaJni, int, facebook::jni::alias_ref<_jintArray*>, int, int, int, facebook::jni::alias_ref<_jstring*>, int, facebook::jni::alias_ref<executorch_jni::ExecuTorchLlamaCallbackJni>, unsigned char>::call(_JNIEnv*, _jobject*, _jintArray*, int, int, int, _jstring*, int, facebook::jni::detail::JTypeFor<executorch_jni::ExecuTorchLlamaCallbackJni, facebook::jni::JObject, void>::_javaobject*, unsigned char)+40) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A        #14 pc 00000000003571b4  /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/base.apk (org.pytorch.executorch.LlamaModule.generate+0)
2024-09-24 17:48:01.625 30912-30912 DEBUG                   crash_dump64                         A        #19 pc 0000000000006ac0  /data/app/~~aG35emmtdavwdMSIKqTV1Q==/com.example.executorchllamademo-STaUGI-KX-56qOQ9kpokqw==/base.apk (com.example.executorchllamademo.MainActivity$4.run+0)
---------------------------- PROCESS ENDED (30915) for package com.example.executorchllamademo ----------------------------
---------------------------- PROCESS ENDED (29659) for package com.example.executorchllamademo ----------------------------

Any idea why this is happening?

Versions

PyTorch version: 2.5.0 Is debug build: False CUDA used to build PyTorch: None ROCM used to build PyTorch: N/A

OS: macOS 14.4.1 (arm64) GCC version: Could not collect Clang version: 15.0.0 (clang-1500.3.9.4) CMake version: version 3.30.3 Libc version: N/A

Python version: 3.10.0 (default, Mar 3 2022, 03:54:28) [Clang 12.0.0 ] (64-bit runtime) Python platform: macOS-14.4.1-arm64-arm-64bit Is CUDA available: False CUDA runtime version: No CUDA CUDA_MODULE_LOADING set to: N/A GPU models and configuration: No CUDA Nvidia driver version: No CUDA cuDNN version: No CUDA HIP runtime version: N/A MIOpen runtime version: N/A Is XNNPACK available: True

CPU: Apple M1

Versions of relevant libraries: [pip3] executorch==0.4.0a0+759e0c8 [pip3] numpy==1.26.4 [pip3] torch==2.5.0 [pip3] torchaudio==2.5.0 [pip3] torchsr==1.0.4 [pip3] torchvision==0.20.0 [conda] executorch 0.4.0a0+759e0c8 pypi_0 pypi [conda] numpy 1.26.4 pypi_0 pypi [conda] torch 2.5.0 pypi_0 pypi [conda] torchaudio 2.5.0 pypi_0 pypi [conda] torchsr 1.0.4 pypi_0 pypi [conda] torchvision 0.20.0 pypi_0 pypi

build settings for the app

i'm following these tutorial pages (note, these are from v0.4.0-rc2 release)

export ANDROID_HOME="/Users/william/Library/Android/sdk"
export ANDROID_NDK="/Users/william/Library/Android/sdk/ndk/25.2.9519653"
export ANDROID_ABI="arm64-v8a"

pushd extension/android                   
   ./gradlew build            
popd
pushd examples/demo-apps/android/LlamaDemo
   ./gradlew :app:setup       
popd
pushd examples/demo-apps/android/LlamaDemo                                            
   ./gradlew :app:installDebug
popd
dvorjackz commented 20 hours ago

cc @kirklandsign

kirklandsign commented 20 hours ago

Hi @grisaitis may I ask which phone are you using and how many RAM is it?

It's not related to app/JNI, but runtime itself.

grisaitis commented 19 hours ago

Sure - Google Pixel 8, with 8GB. Confirmed from my system settings.

Thanks so much for helping troubleshoot this! This library is amazing.

kirklandsign commented 19 hours ago

@grisaitis Thank you for the info! Do you have any lmkd from logcat?

grisaitis commented 6 hours ago

looks like it is a low memory issue, but i'm not sure. these are the lmkd logcat messages right after i click the enter arrow for a prompt:

2024-09-25 10:24:19.378   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.android.apps.scone' (15964), uid 10137, oom_score_adj 500 to free 92212kB rss, 33036kB swap; reason: low watermark is breached and swap is low (712kB < 389076kB)
2024-09-25 10:24:21.869   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.android.carrier' (16153), uid 10113, oom_score_adj 905 to free 61004kB rss, 35280kB swap; reason: low watermark is breached and swap is low (0kB < 389076kB)
2024-09-25 10:24:21.887   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.garmin.android.apps.connectmobile' (15970), uid 10284, oom_score_adj 250 to free 178952kB rss, 32148kB swap; reason: low watermark is breached and swap is low (0kB < 389076kB)
2024-09-25 10:24:22.307   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.android.GoogleCamera' (16267), uid 10121, oom_score_adj 500 to free 107024kB rss, 34436kB swap; reason: low watermark is breached and swap is low (8kB < 389076kB)
2024-09-25 10:24:22.586   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.android.apps.scone' (16277), uid 10137, oom_score_adj 250 to free 90028kB rss, 33128kB swap; reason: low watermark is breached and swap is low (36kB < 389076kB)
2024-09-25 10:24:22.642   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.android.apps.chromecast.app' (16270), uid 10364, oom_score_adj 250 to free 140084kB rss, 34104kB swap; reason: low watermark is breached and swap is low (240kB < 389076kB)
2024-09-25 10:24:22.675   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.process.gservices' (16345), uid 10170, oom_score_adj 250 to free 68600kB rss, 34872kB swap; reason: low watermark is breached and swap is low (776kB < 389076kB)
2024-09-25 10:24:22.725   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.android.projection.gearhead:shared' (16309), uid 10159, oom_score_adj 905 to free 93804kB rss, 34260kB swap; reason: low watermark is breached and swap is low (52kB < 389076kB)
2024-09-25 10:24:22.814   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.android.carrier' (16472), uid 10113, oom_score_adj 905 to free 62372kB rss, 35496kB swap; reason: low watermark is breached and swap is low (0kB < 389076kB)
2024-09-25 10:24:22.815   495-500   lowmemorykiller         lmkd                                 E  process_mrelease 16472 failed: No such process
---------------------------- PROCESS STARTED (16387) for package com.example.executorchllamademo ----------------------------
2024-09-25 10:24:22.975   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.android.projection.gearhead:car' (16505), uid 10159, oom_score_adj 905 to free 89220kB rss, 34844kB swap; reason: low watermark is breached and swap is low (7436kB < 389076kB)
2024-09-25 10:24:22.998   495-495   lowmemorykiller         lmkd                                 I  Kill 'com.google.android.apps.wearables.maestro.companion' (16468), uid 10131, oom_score_adj 915 to free 81072kB rss, 34608kB swap; reason: low watermark is breached and swap is low (14652kB < 389076kB)
---------------------------- PROCESS ENDED (16387) for package com.example.executorchllamademo ----------------------------
---------------------------- PROCESS ENDED (15155) for package com.example.executorchllamademo ----------------------------

any additional tips appreciated!

grisaitis commented 6 hours ago

PS here are all logcat messages, from adb logcat > logcat_output.txt

logcat_output.txt

i asked claude to analyze the output. it suggested the error was originating in the Runner::generate call in the native library, highlighting these three lines from the logcat output. 

09-25 10:32:39.208 20618 20618 F DEBUG   :       #01 pc 0000000001b23e94  /data/app/~~-Z1bTFwXFcIAkEa8u0899w==/com.example.executorchllamademo-HVqGfQ-pjnutaF5__CmORw==/lib/arm64/libexecutorch.so (et_pal_abort+8) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)
09-25 10:32:39.208 20618 20618 F DEBUG   :       #02 pc 0000000001b23ddc  /data/app/~~-Z1bTFwXFcIAkEa8u0899w==/com.example.executorchllamademo-HVqGfQ-pjnutaF5__CmORw==/lib/arm64/libexecutorch.so (executorch::runtime::runtime_abort()+8) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)
09-25 10:32:39.208 20618 20618 F DEBUG   :       #03 pc 0000000001aa15d0  /data/app/~~-Z1bTFwXFcIAkEa8u0899w==/com.example.executorchllamademo-HVqGfQ-pjnutaF5__CmORw==/lib/arm64/libexecutorch.so (example::Runner::generate(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&, int, std::__ndk1::function<void (std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&)>, std::__ndk1::function<void (executorch::extension::llm::Stats const&)>, bool)+2888) (BuildId: c27814f605859baae5b074437bc1e98d565983ea)

any additional tips appreciated. perhaps i should build the executorch library with logging enabled? eg with -DEXECUTORCH_ENABLE_LOGGING=1 flags added to the cmake calls.