Visual Voice Activity Detection (VVAD) integration including live camera and prerecorded demo - Githubissues

oarriaga / paz

Hierarchical perception library in Python for pose estimation, object detection, instance segmentation, keypoint estimation, face recognition, etc.

https://oarriaga.github.io/paz/

MIT License

617 stars 102 forks source link

Visual Voice Activity Detection (VVAD) integration including live camera and prerecorded demo #330

Closed cedric-cfk closed 5 months ago

cedric-cfk commented 8 months ago

This pull request includes:

an integration for the dataset VVAD-LRS3
a model presented by VVAD-LRS3 for binary 3D classification tasks
four different CNN2Plus1D models for binary 3D classification tasks
a detection pipeline for the VVAD task
two demo files using the VVAD pipeline. One using the live camera feed and the second using a prerecorded video file

Note:

[x] There are two TODOs included in the code, regarding the downloading of model files. Those will be resolved in the next days.