search

johndpope / Emote-hack

Emote Portrait Alive - using ai to reverse engineer code from white paper. (abandoned)
https://github.com/johndpope/VASA-1-hack
172 stars 9 forks source link

Data Preparation: #1

Closed johndpope closed 8 months ago

johndpope commented 8 months ago

Download the required datasets. Write data loaders and preprocessors as per the paper's specifications.

150 million images. 250 hours of video

johndpope commented 8 months ago

let's use this https://academictorrents.com/details/843b5adb0358124d388c4e9836654c246b988ff4

johndpope commented 8 months ago

training code on celebhq-v dataset (not using stable diffusion under the hood) https://github.com/JialeTao/MRFA

CHK-0000 commented 8 months ago

Does the 150 million image data used for EMO require images with only human faces? Or do you need images of faces from multiple angles? If not, please let me know. Thank you.

johndpope commented 8 months ago

using the 40gb torrent we have lots of mp4s + we have Dataset - we can load the videos + audio - get the frames / audio features.

https://github.com/johndpope/Emote-hack/blob/main/Net.py#L1004

see training stage 1/2/3