search

Mobile-Artificial-Intelligence / maid_llm

maid_llm is a dart implementation of llama.cpp used by the mobile artificial intelligence distribution (maid)
MIT License
48 stars 12 forks source link

Crash occurs on low-level API Android devices #10

Open canluhuang opened 3 months ago

canluhuang commented 3 months ago

I found the crash occur on some low level API（<= 29） Android devices, this is the crash stack： image The stack after mapping： xxx/Android/sdk/ndk/23.1.7779620/toolchains/llvm/prebuilt/darwin-x86_64/lib64/clang/12.0.8/include/bits/stdatomic.h:273 xxx/Android/maid/packages/maid_llm/src/llama_cpp/src/llama.cpp:18617 xxx/Android/maid/packages/maid_llm/src/maid_llm.cpp:33

And I found that this was caused by turning on an arm64-v8a compilation optimization flag in "maid_llm/android/CMakeLists.txt". image

What will happen if this flag is not enabled?

danemadsen commented 3 months ago

I found the crash occur on some low level API（<= 29） Android devices, this is the crash stack： image The stack after mapping： xxx/Android/sdk/ndk/23.1.7779620/toolchains/llvm/prebuilt/darwin-x86_64/lib64/clang/12.0.8/include/bits/stdatomic.h:273 xxx/Android/maid/packages/maid_llm/src/llama_cpp/src/llama.cpp:18617 xxx/Android/maid/packages/maid_llm/src/maid_llm.cpp:33

And I found that this was caused by turning on an arm64-v8a compilation optimization flag in "maid_llm/android/CMakeLists.txt". image

What will happen if this flag is not enabled?

From memory it will work without that enabled but it makes it VERY slow.