search

google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://ai.google.dev/edge/mediapipe
Apache License 2.0
27.23k stars 5.13k forks source link

How to do batch processing in desktop. #2007

Closed I-aashishM closed 3 years ago

I-aashishM commented 3 years ago

I have seen this issue #452 . According to discussion, I guess we can add batch processing on Desktop.

My concern is that can anyone tell me how could I do batch processing. Any reference or steps?

sgowroji commented 3 years ago

This will help you https://github.com/google/mediapipe/issues/1667#issuecomment-797851624.

google-ml-butler[bot] commented 3 years ago

Are you satisfied with the resolution of your issue? Yes No

impjdi commented 3 years ago

Apologies, this slipped through.

There is no easy solution for that. There are multiple places where the plumbing needs to happen, and I'm not sure what the status is.

The first is the batch dimension support at mediapipe::Tensor. I'm not sure whether the batch dimension is supported by OpenGlTexture2DView and if not, someone would have to do the plumbing on the MediaPipe side. And AFAIK it's not on the priority list.

Then is the support on the TensorFlow Lite GPU side. I am not sure about the latest status of batch dimension support on OpenGL and OpenCL. I remember that we used batching for OpenGL internally for a partial plumbing that never made it to any launch, and thus, I'm not sure whether the code was actually submitted or not. But IIRC, we have some implementation where the shader program is enqueued multiple times along the batch dimension, and thus, you should be able to trigger it. On the OpenCL side, the owner is currently out of office and I don't know what the status is.

Unfortunately no references or documentation, there is no use case or anyone who asked (well, internally). When we do add that, it will be reflected on BlazePose with hand detection, because you could batch hand detection and hand meshes easily in the batching. But until that is updated, I guess no reference :(