search

Picovoice / picollm

On-device LLM Inference Powered by X-Bit Quantization
https://picovoice.ai/
Apache License 2.0
155 stars 6 forks source link

picoLLM Issue: Cannot load model in android 14 (pixel 7a) #40

Closed ishanvohra2 closed 1 month ago

ishanvohra2 commented 1 month ago

Have you checked the docs and existing issues?

SDK

Android

picoLLM package version

1.0.0

Framework version

Android

Platform

Android

OS/Browser version

Android 14.0

Describe the bug

While calling PicoLLM.Builder().setAccessKey(accessKey).setModelPath(modelPath).build(), the library is throwing an error saying that it cannot load the specified model. The stacktrace is as follows:


2024-07-15 10:11:56.037  8449-8449  System.err              com.ishanvohra2.localai              W  ai.picovoice.picollm.PicoLLMIOException: Initialization failed:
2024-07-15 10:11:56.037  8449-8449  System.err              com.ishanvohra2.localai              W    [0] Failed to open file `/sdcard/Download/phi2-290.pllm`.
2024-07-15 10:11:56.038  8449-8449  System.err              com.ishanvohra2.localai              W      at ai.picovoice.picollm.PicoLLMNative.init(Native Method)
2024-07-15 10:11:56.038  8449-8449  System.err              com.ishanvohra2.localai              W      at ai.picovoice.picollm.PicoLLM.<init>(PicoLLM.java:108)
2024-07-15 10:11:56.038  8449-8449  System.err              com.ishanvohra2.localai              W      at ai.picovoice.picollm.PicoLLM.<init>(PicoLLM.java:21)
2024-07-15 10:11:56.038  8449-8449  System.err              com.ishanvohra2.localai              W      at ai.picovoice.picollm.PicoLLM$Builder.build(PicoLLM.java:419)
2024-07-15 10:11:56.038  8449-8449  System.err              com.ishanvohra2.localai              W      at com.ishanvohra2.localai.ui.activities.IntroActivity.onCreate(IntroActivity.kt:48)
2024-07-15 10:11:56.038  8449-8449  System.err              com.ishanvohra2.localai              W      at android.app.Activity.performCreate(Activity.java:8960)
2024-07-15 10:11:56.038  8449-8449  System.err              com.ishanvohra2.localai              W      at android.app.Activity.performCreate(Activity.java:8938)
2024-07-15 10:11:56.038  8449-8449  System.err              com.ishanvohra2.localai              W      at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1526)
2024-07-15 10:11:56.038  8449-8449  System.err              com.ishanvohra2.localai              W      at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3975)
2024-07-15 10:11:56.038  8449-8449  System.err              com.ishanvohra2.localai              W      at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:4173)
2024-07-15 10:11:56.039  8449-8449  System.err              com.ishanvohra2.localai              W      at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:114)
2024-07-15 10:11:56.039  8449-8449  System.err              com.ishanvohra2.localai              W      at android.app.servertransaction.TransactionExecutor.executeNonLifecycleItem(TransactionExecutor.java:231)
2024-07-15 10:11:56.039  8449-8449  System.err              com.ishanvohra2.localai              W      at android.app.servertransaction.TransactionExecutor.executeTransactionItems(TransactionExecutor.java:152)
2024-07-15 10:11:56.039  8449-8449  System.err              com.ishanvohra2.localai              W      at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:93)
2024-07-15 10:11:56.039  8449-8449  System.err              com.ishanvohra2.localai              W      at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2595)
2024-07-15 10:11:56.039  8449-8449  System.err              com.ishanvohra2.localai              W      at android.os.Handler.dispatchMessage(Handler.java:107)
2024-07-15 10:11:56.039  8449-8449  System.err              com.ishanvohra2.localai              W      at android.os.Looper.loopOnce(Looper.java:232)
2024-07-15 10:11:56.039  8449-8449  System.err              com.ishanvohra2.localai              W      at android.os.Looper.loop(Looper.java:317)
2024-07-15 10:11:56.039  8449-8449  System.err              com.ishanvohra2.localai              W      at android.app.ActivityThread.main(ActivityThread.java:8592)
2024-07-15 10:11:56.039  8449-8449  System.err              com.ishanvohra2.localai              W      at java.lang.reflect.Method.invoke(Native Method)
2024-07-15 10:11:56.040  8449-8449  System.err              com.ishanvohra2.localai              W      at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:580)
2024-07-15 10:11:56.040  8449-8449  System.err              com.ishanvohra2.localai              W      at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:878)```

### Steps To Reproduce

1. Follow the instructions in the [docs](https://picovoice.ai/docs/quick-start/picollm-android/)
2. Download the phi2-290 model from picovoice console
3. Set the path of the model file in the downloads folder
4. Build the app

### Expected Behavior

The model should load properly since there is no permission issue while accessing shared files.
albho commented 1 month ago

It looks like your application is having trouble finding the model in the specified file path. Can you make sure that the model file exists in the specified path, and there aren't any typos?

If that's not the issue, please also give our android demo a try and see if you can reproduce the issue.

albho commented 1 month ago

Note: as demonstrated in our android demo here, the model file needs to be copied over to the application directory, since Android 11+ cannot directly open files from Downloads or any other external storage directory.

I'm going to close this issue for now, please feel free to re-open if the issue persists.