[EdgeTPU Support]: Possible to use Google Mediapipe model(s)?

[cvocvo]

Describe the problem you are having

Is it possible to use Google's Mediapipe model for face detection?

Github project: https://github.com/google/mediapipe More info: https://github.com/google/mediapipe/blob/master/docs/solutions/face_detection.md What I think is the Google Coral model to use? https://github.com/google/mediapipe/blob/master/mediapipe/examples/coral/models/face-detector-quantized_edgetpu.tflite

Ideally, I'd like to run face detection as a second model on my google coral in addition to the built-in object detection model. If that's not possible, I'd like to just get face detection working as a binary yes/no we've detected a face or haven't detected a face. Then with the home assistant integration, I could run automations based on either face detected or not detected.

Version

v12 Beta 8

Frigate config file

Not sure how to configure this to override or specify a second model

docker-compose file or Docker CLI command

Standard setup; not doing anything fancy other than passing the USB for the coral device through to the container.

Relevant log output

N/A (haven't gotten this far)

Operating system

Debian

Install method

Docker Compose

Coral version

USB

Any other information that may be helpful

(If you need the docker-compose or frigate config files I can pull those later today, but I'm generally just not sure how to start on this or if it's even possible.)

[NickM-27]

Face recognition is recommended to be done through https://github.com/jakowenko/double-take right now. Currently, Frigate does not support running models with different labelmaps / sizes so this won't work.

Frigate+ models will also support face detection as a label: https://deploy-preview-4055--frigate-docs.netlify.app/integrations/plus

[cvocvo]

By the way, I've seen more complicated setups with frigate + deepstack + double-take. However, I don't need/want facial recognition and I just need facial detection.

[cvocvo]

Currently, Frigate does not support running models with different labelmaps / sizes so this won't work.

Is this comment with respect to the Google Mediapipe face detection model? (I'm way new to this so I don't understand the relationship of labelmaps to frigate. I don't know if the mediapipe face detection model needs a labelmap?)

Frigate+ models will also support face detection as a label: https://deploy-preview-4055--frigate-docs.netlify.app/integrations/plus

Gotcha, is there an ETA or beta version of face detection I can buy access to? That seems like it would be the easiest option for me (and avoid the need to set up deepstack + double-take).

Thank you!

[kaytwo]

Running multiple models simultaneously on the EdgeTPU is somewhat complicated, see the documentation. This would require either merging both models so that they can fit into the 8MB RAM of the EdgeTPU (which might lead to lower accuracy), or accepting the additional overhead of swapping between the two models for each detection task that's run when motion is detected.

The Frigate+ approach (training with "face" as an additional label in the object detection model) or the external classifier (double-take) approach are probably your best options at this point.