Design for preprocessor at scale / distributed

[johndpope]

BACKGROUND so current thinking is - I'm GPU poor - there's community members who have a spare gpu around who want to help. Need to preprocess some videos - but it's very time consuming. We have 35,000 videos magnet:?xt=urn:btih:843b5adb0358124d388c4e9836654c246b988ff4&dn=CelebV-HQ&tr=https%3A%2F%2Facademictorrents.com%2Fannounce.php&tr=https%3A%2F%2Fipv6.academictorrents.com%2Fannounce.php

each one taking 1-5+ mins to process. This code is in the EmoDataset loaderr

it achieves

• frame dumping via decord (needs redoing with VALI or torchaudio.StreamReader https://github.com/dmlc/decord/issues/283)
• warping / thin spline - it has parameter for dialin this up / down.
• auto-cropping / sweet spot
• background removal (option for greenscreen)
• normalization
• resizing to 512x512
• cache into a single npz file for subsequent training runs that does all this

SOLUTION 1) make huggingface repo that contains just the npz files 2) setup organziation that people can join https://huggingface.co/organizations/covershot/share/jVGQxiItlPNKPUbwcSXfMEICDLJxLttyAB 3) allow users to upload /commit npz file to repo.

[elihalpern]

I'm willing to assist! Would love to see this implemented. I have a 3090 and a 4060 TI.

[johndpope]

looks like pillow doesn't have gpu support. https://github.com/python-pillow/Pillow/issues/1546

i got multicore to send work to multicpus / which does speed things up - but it never completes and gets to next line - something not right. https://github.com/johndpope/MegaPortrait-hack/tree/feat/38-multicore

https://github.com/johndpope/MegaPortrait-hack/blob/feat/38-multicore/EmoDataset.py#L251

save the date