search

microsoft / onnxruntime-inference-examples

Examples for using ONNX Runtime for machine learning inferencing.
MIT License
1.16k stars 331 forks source link

Android Example phi-3 error: failed:This is an invalid model #437

Closed psj2202 closed 4 months ago

psj2202 commented 4 months ago

Hi,

I'm getting into the crash issue when launching the app with showing below error in the platform log. I've tried to figure out how to resolve the issue and found out below article but it's still showing same error with a fix.

https://github.com/microsoft/onnxruntime-inference-examples/pull/435

05-29 18:35:17.122 10336 643 643 E AndroidRuntime: FATAL EXCEPTION: main 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: Process: ai.onnxruntime.genai.demo, PID: 643 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: java.lang.RuntimeException: Unable to start activity ComponentInfo{ai.onnxruntime.genai.demo/ai.onnxruntime.genai.demo.MainActivity}: java.lang.RuntimeException: ai.onnxruntime.genai.demo.GenAIException: Load model from /data/user/0/ai.onnxruntime.genai.demo/files/phi3-mini-4k-instruct-cpu-int4-rtn-block-32-acc-level-4.onnx failed:This is an invalid model. Type Error: Type 'tensor(float)' of input parameter (/model/layers.0/attn/qkv_proj/MatMul/output_0) of operator (GroupQueryAttention) in node (/model/layers.0/attn/GroupQueryAttention) is invalid. 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:4165) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:4323) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:103) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:139) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:96) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2686) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at android.os.Handler.dispatchMessage(Handler.java:106) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at android.os.Looper.loopOnce(Looper.java:230) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at android.os.Looper.loop(Looper.java:319) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at android.app.ActivityThread.main(ActivityThread.java:8979) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at java.lang.reflect.Method.invoke(Native Method) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:588) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1103) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: Caused by: java.lang.RuntimeException: ai.onnxruntime.genai.demo.GenAIException: Load model from /data/user/0/ai.onnxruntime.genai.demo/files/phi3-mini-4k-instruct-cpu-int4-rtn-block-32-acc-level-4.onnx failed:This is an invalid model. Type Error: Type 'tensor(float)' of input parameter (/model/layers.0/attn/qkv_proj/MatMul/output_0) of operator (GroupQueryAttention) in node (/model/layers.0/attn/GroupQueryAttention) is invalid. 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at ai.onnxruntime.genai.demo.MainActivity.onCreate(MainActivity.java:51) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at android.app.Activity.performCreate(Activity.java:9001) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at android.app.Activity.performCreate(Activity.java:8970) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1456) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:4147) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: ... 12 more 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: Caused by: ai.onnxruntime.genai.demo.GenAIException: Load model from /data/user/0/ai.onnxruntime.genai.demo/files/phi3-mini-4k-instruct-cpu-int4-rtn-block-32-acc-level-4.onnx failed:This is an invalid model. Type Error: Type 'tensor(float)' of input parameter (/model/layers.0/attn/qkv_proj/MatMul/output_0) of operator (GroupQueryAttention) in node (/model/layers.0/attn/GroupQueryAttention) is invalid. 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at ai.onnxruntime.genai.demo.GenAIWrapper.loadModel(Native Method) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at ai.onnxruntime.genai.demo.GenAIWrapper.(GenAIWrapper.java:22) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at ai.onnxruntime.genai.demo.MainActivity.createGenAIWrapper(MainActivity.java:184) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at ai.onnxruntime.genai.demo.MainActivity.downloadModels(MainActivity.java:162) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: at ai.onnxruntime.genai.demo.MainActivity.onCreate(MainActivity.java:48) 05-29 18:35:17.122 10336 643 643 E AndroidRuntime: ... 16 more

Any idea how I can get over it?

edgchen1 commented 4 months ago

435 was merged. Please delete any previously downloaded files first and try again.

psj2202 commented 4 months ago

confirmed that it's working with latest change. Thanks!