megvii-research / megactor

Apache License 2.0
581 stars 78 forks source link

有人一起跑吗,怎么感觉效果一般? #12

Open ali762111 opened 1 week ago

ali762111 commented 1 week ago

I think,感觉跑完以后效果没有那么好,方法有点熟悉,就像是缝合了AniPortrait和EMO的story postpoint和referencenet的desc比较熟悉 And generation deatils 的抖动有点明显 有后续有新版本可以戳一下......

frankjiang commented 1 week ago

同尝试了一下,效果有点惊悚。。。

观察到的问题如下:

  1. 五官偏离本人
  2. 面部扭曲,眼睛鼻子眉毛都歪了(开了 --contour-preserve)
  3. 抖动明显

参考 X-Portrait 增加关键点控制可能会好一些?

lhd777 commented 1 week ago

感谢大家的使用体验和反馈建议,目前我们的方法在运动幅度较小的场景能够生成自然流畅的视频生成结果,如主页的视频生成结果所示。然而如大家所说,我们内部测试Megactor仍然存在limitation:泛化性不足,细节生成的一致性不足等等。我们内部正在进一步优化Megactor,已经有一定的改进,欢迎大家关注我们的后续改进版本。

Thank you all for your valuable feedback and insights based on your usage experiences. Currently, our method is capable of generating natural and smooth video synthesis results in scenarios with minimal movements, as demonstrated by the video generation examples on our homepage. However, as many of you have pointed out, our internal tests reveal several limitations of Megactor, including inadequate generalization capability and inconsistent detail generation, among others. We are actively working internally to further optimize Megactor and have already made notable improvements. We warmly welcome you to stay tuned for our upcoming enhanced versions.

addvaluejack commented 1 week ago

可能预训练数据里女性比例高的原因?川普的测试结果里给加上了假睫毛 7

https://github.com/megvii-research/megactor/assets/3754345/bb31fa7f-a40e-434c-9d72-86c31b381b0f

theoldsong commented 6 days ago

可能预训练数据里女性比例高的原因?川普的测试结果里给加上了假睫毛 7

7.mp4

可能预训练数据里女性比例高的原因?川普的测试结果里给加上了假睫毛 7

7.mp4

请问,只能在linux上运行吗?win支持吗

addvaluejack commented 4 days ago

可能预训练数据里女性比例高的原因?川普的测试结果里给加上了假睫毛 7 7.mp4

可能预训练数据里女性比例高的原因?川普的测试结果里给加上了假睫毛 7 7.mp4

请问,只能在linux上运行吗?win支持吗

你好,我只试过在Linux上运行

lhd777 commented 4 days ago

我们发现了开源代码推理config的inference.yaml的一个Bug。该Bug会导致推理的效果变差。具体的原因是我们采用的frames=16进行了训练,推理config的参数上为frames=8。该Bug已经Fixed,修改部分见 https://github.com/megvii-research/megactor/blob/main/configs/inference/inference.yaml#L23

We discovered a bug in the inference.yaml configuration for open-source code inference, which led to degraded performance. Specifically, the issue arose because we trained the model with frames=16, while the inference configuration was set with frames=8. The bug has been fixed, and the modifications can be seen at the https://github.com/megvii-research/megactor/blob/main/configs/inference/inference.yaml#L23 .