search

johndpope / MegaPortrait-hack

Using Claude Opus to reverse engineer code from MegaPortraits: One-shot Megapixel Neural Head Avatars
https://arxiv.org/abs/2207.07621
42 stars 7 forks source link

Stage 1 - training losses - todo contrastive_loss #14

Closed johndpope closed 3 weeks ago

johndpope commented 1 month ago

https://github.com/johndpope/MegaPortrait-hack/blob/main/train.py#L196

adding any of the others hits a bug with gradients.

Jie-zju commented 4 weeks ago

Have you noticed "losses are calculated using only foreground regions" mentioned in Paper? I tried implement the network (include somewhat different from paper ) and train it. However, just use loss on all images for training seems fail, since the training stage is not stable. I think "losses are calculated using only foreground regions" May Be Important ! Please attention on it!

johndpope commented 4 weeks ago

thanks for pointing this out. I fix later.

johndpope commented 4 weeks ago

i draft this - yet to test it out. https://github.com/johndpope/MegaPortrait-hack/pull/21/files

so much flux - needs stabilising. fyi @robinchm

I look at the patchgan stuff

johndpope commented 4 weeks ago

warped_driving_frame:torch.Size([3, 129, 129])

on my training branch - i implement and fix - but now the warped / cropped image is small.... i can make it bigger - or I have to work out how the network can accept smaller images.... not sure. https://github.com/johndpope/MegaPortrait-hack/tree/feat/14-training

johndpope commented 3 weeks ago

Screenshot from 2024-05-30 12-15-54

some progress.