search

georgesterpu / avsr-tf1

Audio-Visual Speech Recognition using Sequence to Sequence Models
GNU General Public License v3.0
81 stars 28 forks source link

Inquiry about aus csv generation #20

Closed zzgg1 closed 4 years ago

zzgg1 commented 4 years ago

Hello, I found we need csv files to use append aus in your code, do you mind if share how to generate aus csv files?

georgesterpu commented 4 years ago

Hi @zzgg1

I have used the pre-trained models from the OpenFace toolkit to estimate the Action Units intensities

The csv files are generated by OpenFace when the extract_faces.py script is used. According to the documentation of OpenFace, the action units should be enabled by default, and can also be enforced with the -aus flag.

If you have a different way to preprocess your video data other than with OpenFace, you would have to replace this function that parses the AU annotations.

Let me know if this works for you.

zzgg1 commented 4 years ago

Thanks, I would give it a try