search

josephrocca / openai-clip-js

OpenAI's CLIP model ported to JavaScript using the ONNX web runtime
MIT License
137 stars 10 forks source link

Errors when running demo #2

Closed nbardy closed 2 years ago

nbardy commented 2 years ago

I'm getting an error when running the text model in chrome.

Screen Shot 2022-04-29 at 9 24 31 AM

The image model runs successfully on wasm, but fails on webgl

Screen Shot 2022-04-29 at 9 25 49 AM
nbardy commented 2 years ago

The CLIP search demo works, so it must just be an issue with the images

josephrocca commented 2 years ago

Ah yeah sorry the WebGL backend doesn't work due to conversion issues (I've added a message now to point that out). Not sure how much extra model conversion fiddling it would take to get it going since it's a serial chain so I don't know if there's 10 more to solve after fixing that one. In general though it always ends up being a lot harder to get WebGL working compared to Wasm. Wasm has more op support because they compile the C/C++ runtime directly in case of onnx and tflite. Hoping that when we get WebGPU, it'll have high op coverage for the same reason as Wasm - guessing it should be possible to directly port the GPU kernels.

Problem with the text model should be fixed now too. Thanks! (feel free to re-open if it's still not working for you)

nbardy commented 2 years ago

I ran into the same serial issue thing and just gave up. Kudos for getting this far