What is the resolution compared to Thin-Plate-Spline-Motion-Model

[FurkanGozukara]

I think yours is same as Thin-Plate-Spline-Motion-Model

Their resolution was like 256px which was making it extremely useless

I wonder what is your resolution?

here a test i made with Thin Plate Spline

https://github.com/MBZUAI-Metaverse/VOODOO3D-official/assets/19240467/06144a0c-d064-492d-b799-0879d4d3793b

[P0lyFish]

Hi, thank you for your interest! To answer your question, our model produces 512px resolution images.

While both methods (Our method and the one you linked) are designed for the facial reenactment task, there are some key differences in terms of techniques and features:

• Our method is 3D-aware and therefore can produce very consistent results across different viewing angles which is not doable with 2D-based models to the best of my knowledge. In addition, we can extract 3D information from the results (e.g. depth, mesh, etc)
• We can takes very extreme-posed face source image (the face that you want to drive) as can be seen in the paper.
• Thin-Plate Spline Motion Model uses keypoints for 2D-warping. As shown in previous works [1], While this approach can be used for full body reenactment, it leads to uncanny expression for the facial reenactment task. In contrast, our model uses the face images directly and hence produce realistic expressions.

Checkout our project page for more results: https://p0lyfish.github.io/voodoo3d/

[1] Drobyshev, Nikita, et al. "Megaportraits: One-shot megapixel neural head avatars." Proceedings of the 30th ACM International Conference on Multimedia. 2022.

[FurkanGozukara]

@P0lyFish thank you so much for reply

Will we able to use yours with pre trained model and code?

[P0lyFish]

The paper for this project is under submission. We will release all the code and pretrained models (including our reimplementation of MegaPortraits and Lp3D which are closed source) after the acceptance of the paper.

[FurkanGozukara]

The paper for this project is under submission. We will release all the code and pretrained models (including our reimplementation of MegaPortraits and Lp3D which are closed source) after the acceptance of the paper.

Sad. I am a Computer Engineer and my submissions were taking like a year to get accepted. 1 year means forever in AI

[P0lyFish]

Wouldn't take a whole year. I think the paper decision will be released at the end of January 2024. So the code would be available around that time too.

[FurkanGozukara]

Wouldn't take a whole year. I think the paper decision will be released at the end of January 2024. So the code would be available around that time too.

I hope so too be fast

[FurkanGozukara]

AliBaba released DreamTalk and it is really really low quality and they have forced watermark lol

https://github.com/MBZUAI-Metaverse/VOODOO3D-official/assets/19240467/786f6d66-8193-41b4-9681-0facea342140

[JZArray]

The paper for this project is under submission. We will release all the code and pretrained models (including our reimplementation of MegaPortraits and Lp3D which are closed source) after the acceptance of the paper.

@P0lyFish could you release codes for MegaPortraits?