Open Qifeng-Wu99 opened 7 months ago
Hi Qifeng, I suspect it might be the problem of selecting a appropriate checkpoint of the postnet. Maybe you can refer to this doc and this figure. Also, we plan to release GeneFace++ in Feb. 2024, which could well handle the challenge to hand-pick the postnet
Appreciate the fantastic job done by the authors.
As I am trying to reproduce the result in the demo released by the authors, I train the sync net, audio to motion generator, post net and rad nerf from scratch on my own with the hyper-parameters released with the code.
However, the lip alighnment with the audio in my result is not satisfactory when compared to that obtained by the authors.
I wonder if there is some trick to tune/refine the hyper parameters to achieve better results.
Thanks in advance.
感谢作者的精彩工作。
在尝试复现作者发布的演示结果时,我使用随代码一起发布的超参数,从头开始训练syncnet、audio2motion generator、postnet和rad nerf。
然而,与作者获得的结果相比,我得到的结果中唇部与音频的对齐并不令人满意。
想问下大家能否分享一下如何调整超参数从而获得更好的效果。
谢谢。
您好,这个对齐的问题您找到了吗?我在用其他视频进行syncnet、audio2motion generator、postnet和rad nerf训练也发现训练出来的模型嘴形对不齐。我在考虑在用Hubert提取特征时候是不是对应不同语种使用不同的hubert
Appreciate the fantastic job done by the authors.
As I am trying to reproduce the result in the demo released by the authors, I train the sync net, audio to motion generator, post net and rad nerf from scratch on my own with the hyper-parameters released with the code.
However, the lip alighnment with the audio in my result is not satisfactory when compared to that obtained by the authors.
I wonder if there is some trick to tune/refine the hyper parameters to achieve better results.
Thanks in advance.
感谢作者的精彩工作。
在尝试复现作者发布的演示结果时,我使用随代码一起发布的超参数,从头开始训练syncnet、audio2motion generator、postnet和rad nerf。
然而,与作者获得的结果相比,我得到的结果中唇部与音频的对齐并不令人满意。
想问下大家能否分享一下如何调整超参数从而获得更好的效果。
谢谢。