search

google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://mediapipe.dev
Apache License 2.0
26.27k stars 5.05k forks source link

MP Vision Tasks web solutions performance... #5377

Open Honya2000 opened 2 months ago

Honya2000 commented 2 months ago

Hello,

I recently compiled WASM mediapipe build from the sources publicly available. It worked fine for ImageSegmentation tasks with CPU SIMD backend. But unfortunately later I realized it cannot be used for GPU inferencing because there are no sources available for GLES30 backend.

So I had to stick to standard task-vision API wasm NPM builds. But unfortunately standard library is about twice slower than my own compiled wasm module. My own module inferencing time is 4ms, standard library takes 8ms. I suspect - problem is in multiple sync points between CPU and GPU in standard library with CPU backend. Not sure why standard API implementation always uses texture as an input. During WEB profiling I see it uses glReadPixels within wasm module and it is slow! In my own implementation I never use WebGL if CPU backend is choosed. So why default API uses webgl2 based canvas ?

Honya2000 commented 2 months ago

Any updates regarding this issues? The situation is even worse. Today i tested the model which takes 5 ms with my own compiled WASM module. But it takes over 22ms with default tasks vision solution. So default wasm is almost 5 times slower!

Honya2000 commented 2 months ago

Profiled in chrome. glReadPixels takes over 10 ms in wasm module... With CPU backend!!!