Why not use the newer & improved MetaPortrait instead of reimplementing MegaPortrait?

[mrT333]

Did you know about MetaPortrait?

https://github.com/Meta-Portrait/MetaPortrait

It's an improved version over MegaPortrait but it already has pretrained weights released, as well as inference and training code for further finetuning!

In their paper they cite MegaPortrait as one of the Inspirations that they improved upon: https://arxiv.org/pdf/2212.08062

Is there a reason why you put so much effort into implementing the older & inferior MegaPortrait instead of using or finetuning the newer MetaPortrait?

[johndpope]

Fantastic question. To be upfront - ( the code + trained models + new DATASET) that Samsung AI labs - with one of the key authors neeek2303 is being upgraded and released in July thanks to Facebook. https://github.com/neeek2303/EMOPortraits So all these efforts are mute regardless.

I'm attempting to recreate the VASA-1 paper - https://github.com/johndpope/vasa-1-hack

Digging through that 2024 paper - it says that 71 - MetaPortraits 2023 falls short - and they base VASA on the OLDER paper MegaPortraits 17

Microsoft 2023 - @ChenyangQiQi @ForeverFancy built out MetaPortrait without the code from Samsung - and it seems like they either couldn't re-implement the warping code leveraging the resnet stack. Surprisingly when I built out the code it was a complete surprise that within the resnet backbone - there's pitch / yaw / roll variables out of the box with inference at lower layers. The densemotionnetwork is computational expensive so - it's not going to yield high fps. I did trawl through hundreds of listing of github results for the resnet50 code specific to this use case - but it doesn't seem like anyone in the world has actually implemented it (yet).

The details / accuracy of this code recreation - jury is still out.

[Jie-zju]

Did you know about MetaPortrait?

https://github.com/Meta-Portrait/MetaPortrait

It's an improved version over MegaPortrait but it already has pretrained weights released, as well as inference and training code for further finetuning!

In their paper they cite MegaPortrait as one of the Inspirations that they improved upon: https://arxiv.org/pdf/2212.08062

Is there a reason why you put so much effort into implementing the older & inferior MegaPortrait instead of using or finetuning the newer MetaPortrait?

How about MetaPortrait on custom dataset training under the lack of pretrained weights on LADMAK ENCODER AND IDENTITY ENCODER? Current not found an acceptable way instead.

[johndpope]

@Jie-zju - not sure what you mean. I may look at the high res componennt next from MetaPortrait to upgrade this codebase - it has some spade models. for now - I had a breakthrough with the training code - this is first epoch. looks like it's working.