search

microsoft / onnxruntime-inference-examples

Examples for using ONNX Runtime for machine learning inferencing.
MIT License
1.16k stars 331 forks source link

PHI3 onnxruntime-web "Live Demo here" fails . No questions answered. #431

Open alanscot opened 4 months ago

alanscot commented 4 months ago

PHI3 Fail AGAIN The loads completed successfully. But empty answer boxes were all that I saw. :( Chrome v122 on Windows 11, RAM 32gb, NVDIA RTX4070 8gb Alan

alanscot commented 4 months ago

SIMILARLY: <<<<<<<<<<<<<<<<<<< No error messages appear on https://huggingface.co/spaces/Xenova/experimental-phi3-webgpu in Chrome - Windows 11

It does display 1.29tokens/second every few seconds that changes A minute later its 1.39tokens/second then 1.38tokens/second 10 secs later its 1.35 tokens/second then 1.34tokens/second and back to 1.35tokens/second then 1.33 About 7 mins later its "Generated 513 tokens in 382.79 seconds (1.34tokens/second)" which seems to be the end of changes. I ask another Q and still nothing but empty answer box. And ever changing -- ( starting with) "1.30 tokens/second" again But this time it only takes about 3 minutes to end with "Generated 513 tokens in 175.92 seconds (2.92tokens/second)" Then I ask a 3rd Q -- same thing that ends with Generated 513 tokens in 127.52 seconds (4.02tokens/second)

PHI3 Fail 1 AlScot 6 days ago

I hit RESET -- The screen clears -- READY appears I ask a 1st question same thing -- ending this time with "Generated 513 tokens in 78.33 seconds (6.55tokens/second)"

satyajandhyala commented 2 weeks ago

Tested successfully on Windows11 with Chrome version Version 128.0.6613.138 (Official Build) (64-bit) and Chrome Canary Version 130.0.6722.0 (Official Build) canary (64-bit).