For all the viewers.

[Kevinfringe]

Hi all, Thanks so much for the support of this implementation of Megaportrait. I made this attempt to reproduce this paper mainly for one of my projects. However, I've been stuck on the implementation of 3D warping mentioned in this paper for quite a while. In short, this version of implementation cannot work so far, and since I've been pretty busy recently, so I might have to leave this repo aside. Anyway, I'm very glad to have your suggestions or guidance on this implementation!

Thanks in advance!

[fmac2000]

Thanks for keeping us up to date Kevin - I'll look around and see if I can pull any people on this. If possible - updating the repo to your latest build would help speed things up.

A message to any others reading: Let's use this thread to discuss and pool resources into this - it'd be a shame to see all the hardwork go to waste when it is so close to the end!

[fmac2000]

Hi Kevin, hope this finds you - here's what I found regarding 3D warping methods, it seems to based from a method referenced from these two papers: "Warping fields synthesized using machine learning (including deep learning)"

https://arxiv.org/abs/1806.06503 https://arxiv.org/abs/1607.07215

And an often used baseline for a warp based approach is a paper named X2Face - https://arxiv.org/pdf/1807.10550.pdf https://github.com/oawiles/X2Face/tree/master/UnwrapMosaic

Hopefully this clears up the confusion? I can look deeper into all the references of MegaPortraits to find a closer match if this doesn't help, let me know what you think. If any researchers could help explain the implementation details I'd much appreciate it, I'm not familiar with this technique.

Below I've linked any papers related to MegaPortraits that has source code with warping generators - thanks again Kevin :)

• https://github.com/saic-violet/bilayer-model (https://arxiv.org/pdf/2008.10174.pdf) [Good explanations on PG 3-4]
• https://github.com/zhanglonghao1992/One-Shot_Free-View_Neural_Talking_Head_Synthesis (https://arxiv.org/pdf/2011.15126.pdf)
• https://github.com/harlanhong/CVPR2022-DaGAN (https://arxiv.org/abs/2203.06605)

[johndpope]

I think Claude OPUS just solved this - https://github.com/Kevinfringe/MegaPortrait/issues/5

UPDATE

• the training code https://github.com/johndpope/MegaPortrait/blob/master/train.py
• and warping code https://github.com/johndpope/MegaPortrait/blob/master/model.py#L359

being rebulit - work in progress.