search

microsoft / onnxruntime-inference-examples

Examples for using ONNX Runtime for machine learning inferencing.
MIT License
1.16k stars 331 forks source link

Android Example phi-3 error: failed:This is an invalid model #434

Closed phil-iu closed 4 months ago

phil-iu commented 4 months ago

Hi! I'm trying to run the Android example app, but I always get the following error:

java.lang.RuntimeException: Unable to start activity ComponentInfo{ai.onnxruntime.genai.demo/ai.onnxruntime.genai.demo.MainActivity}: java.lang.RuntimeException: ai.onnxruntime.genai.demo.GenAIException: Load model from /data/user/0/ai.onnxruntime.genai.demo/files/phi3-mini-4k-instruct-cpu-int4-rtn-block-32-acc-level-4.onnx failed:This is an invalid model. Type Error: Type 'tensor(float)' of input parameter (/model/layers.0/attn/qkv_proj/MatMul/output_0) of operator (GroupQueryAttention) in node (/model/layers.0/attn/GroupQueryAttention) is invalid.

I followed the instructions in the readme. Any ideas how to solve this!

Thank you! :)

devYonz commented 4 months ago

I am unable to run the program on an M2Max AMD mac.Here are a couple of observations

  1. I had to create a custom emulator with 8GB
  2. I tried with 128k model and ran into other issues.
  3. When I opened tokenizer.json manually, it gave me an inconsistent line return warning I am trying to run the app on a mac with AMD and the app starts constantly crashing after downloading tokenizer.json. 
    java.lang.RuntimeException: ai.onnxruntime.genai.demo.GenAIException: [json.exception.parse_error.101] parse error at line 2, column 1: syntax error while parsing value - invalid literal; last read: '<U+000A><U+000E>'
at ai.onnxruntime.genai.demo.ModelDownloader.downloadModel(ModelDownloader.java:62)
at ai.onnxruntime.genai.demo.MainActivity.lambda$downloadModels$0$ai-onnxruntime-genai-demo-MainActivity(MainActivity.java:152)
at ai.onnxruntime.genai.demo.MainActivity$$ExternalSyntheticLambda1.run(Unknown Source:12)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.lang.Thread.run(Thread.java:920)
Caused by: ai.onnxruntime.genai.demo.GenAIException: [json.exception.parse_error.101] parse error at line 2, column 1: syntax error while parsing value - invalid literal; last read: '<U+000A><U+000E>'
at ai.onnxruntime.genai.demo.GenAIWrapper.createTokenizer(Native Method)
at ai.onnxruntime.genai.demo.GenAIWrapper.<init>(GenAIWrapper.java:23)
at ai.onnxruntime.genai.demo.MainActivity.createGenAIWrapper(MainActivity.java:170)
at ai.onnxruntime.genai.demo.MainActivity.access$500(MainActivity.java:23)
at ai.onnxruntime.genai.demo.MainActivity$2.onDownloadComplete(MainActivity.java:158)
cdiddy77 commented 4 months ago

I am getting this same error, straight out of the box. I am running on Samsung Galaxy S23. Same error as @devYonz

devYonz commented 4 months ago

I finally got mine working on the emulator. Try opening /data/data/packagename/files/tokenizer.json through the device explorer in android studio. If it is gibberish you can solve it by downloading that file from huggingface separately and replacing it. It's read-only so you will have to upload into the files directly after deleting it. @cdiddy77

cdiddy77 commented 4 months ago

by

Wow thanks 🙏

cdiddy77 commented 4 months ago

Very curious why is that file corrupted? It looks fine when I hit the download url.

cdiddy77 commented 4 months ago

Turns out it was a very simple bug: https://github.com/microsoft/onnxruntime-inference-examples/pull/435

edgchen1 commented 4 months ago

Fixed by #435.