search

johndpope / MegaPortrait-hack

Using Claude Opus to reverse engineer code from MegaPortraits: One-shot Megapixel Neural Head Avatars
https://arxiv.org/abs/2207.07621
68 stars 7 forks source link

Training set - LMDBDataset #2

Closed johndpope closed 3 months ago

johndpope commented 4 months ago

I think the original codebase is closely related to this https://github.com/SamsungLabs/rome?tab=readme-ov-file

https://github.com/search?q=LMDBDataset&type=code

johndpope commented 4 months ago

B.3 Datasets preprocessing We obtain the VoxCeleb2HQ dataset by first downloading the original videos from the VoxCeleb2 [4 ] dataset. These videos are processed using an off-the-shelf face [41 ] and keypoints [ 2] detectors and cropped frame-by-frame around the head regions. Then, the obtained cropped frames are first filtered by their resolution, t

https://www.google.com/search?q=S3FD%3A+Single+Shot+Scale-Invariant+Face+Detector&oq=S3FD%3A+Single+Shot+Scale-Invariant+Face+Detector&gs_lcrp=EgZjaHJvbWUyBggAEEUYOTIGCAEQRRg60gEHMTkyajBqN6gCALACAA&sourceid=chrome&ie=UTF-8

https://github.com/sfzhang15/SFD

johndpope commented 4 months ago

Screenshot from 2024-05-13 08-14-17

this data prep step feels out of date. were the keypoints calculated per frame?

could a more modern approach be swapped in https://github.com/search?q=repo%3AZejun-Yang%2FAniPortrait%20lmk&type=code

does VASA follow same approach?

johndpope commented 3 months ago

i upgrade the video pre processing to remove background - and other transforms. the ffhq is ok for now.