search

microsoft / onnxruntime-inference-examples

Examples for using ONNX Runtime for machine learning inferencing.
MIT License
1.07k stars 312 forks source link

Updated phi-3 mobile android example #420

Closed YUNQIUGUO closed 2 months ago

YUNQIUGUO commented 2 months ago

This is based on the following two prs:

The GenAI lib used in the current version of this app is built from source code for android from onnxruntime-genai repo and uses the current main version of GenAI C API source code.

https://github.com/microsoft/onnxruntime-genai/pull/227 https://github.com/microsoft/onnxruntime-genai/pull/229

YUNQIUGUO commented 2 months ago

TODO: still a couple commits might be pushed to this branch for clean up/minor updates.

pxz2016 commented 2 months ago

app crash when use chinese: gotNextToken:
2024-05-01 20:28:37.476 9113-9185 GenAI ai.onnxruntime.genai.demo I gotNextToken: 你 2024-05-01 20:28:37.661 9113-9185 GenAI ai.onnxruntime.genai.demo I gotNextToken: 好 2024-05-01 20:28:37.848 9113-9185 GenAI ai.onnxruntime.genai.demo I gotNextToken: ！ 2024-05-01 20:28:38.036 9113-9185 time.genai.demo ai.onnxruntime.genai.demo A java_vm_ext.cc:594] JNI DETECTED ERROR IN APPLICATION: input is not valid Modified UTF-8: illegal continuation byte 0 java_vm_ext.cc:594] string: '?' java_vm_ext.cc:594] input: '0xe5' java_vm_ext.cc:594] in call to NewStringUTF java_vm_ext.cc:594] from java.lang.String ai.onnxruntime.genai.demo.GenAIWrapper.run(long, long, java.lang.String, boolean) 2024-05-01 20:28:38.424 9113-9185 time.genai.demo ai.onnxruntime.genai.demo A runtime.cc:675] Runtime aborting...

ithraasoft-dev commented 2 months ago

Got this error

                                                                                               java.lang.RuntimeException: Unable to start activity ComponentInfo{ai.onnxruntime.genai.demo/ai.onnxruntime.genai.demo.MainActivity}: ai.onnxruntime.genai.demo.GenAIException: Deserialize tensor model.layers.29.attn.o_proj.MatMul.weight_Q4 failed.tensorprotoutils.cc:904 GetExtDataFromTensorProto External initializer: model.layers.29.attn.o_proj.MatMul.weight_Q4 offset: 2466656256 size to read: 4718592 given file_length: 2334412800 are out of bounds or can not be read in full.
YUNQIUGUO commented 2 months ago

Got this error

                                                                                               java.lang.RuntimeException: Unable to start activity ComponentInfo{ai.onnxruntime.genai.demo/ai.onnxruntime.genai.demo.MainActivity}: ai.onnxruntime.genai.demo.GenAIException: Deserialize tensor model.layers.29.attn.o_proj.MatMul.weight_Q4 failed.tensorprotoutils.cc:904 GetExtDataFromTensorProto External initializer: model.layers.29.attn.o_proj.MatMul.weight_Q4 offset: 2466656256 size to read: 4718592 given file_length: 2334412800 are out of bounds or can not be read in full.

Are you trying this on an actual device or in android emulator?