讨论一下 DINet微调的问题

[tailangjun]

DINet要训练的模型有 5个：Syncnet、Frame64、Frame128、Frame256和Clip256。如果想在一组底模上微调，需要微调哪几个模型呢？

• Syncnet比较独立，直接基于底模 Syncnet微调就可以了
• Frame采用的是 coarse-to-fine训练方式，我觉得 Frame64直接基于 Frame256底模来微调就可以了，后来又发现 Frame和 Clip256的 net_g也是同一份，感觉 Frame64基于 Clip256底模来微调可能更好，这样可以保留底模中更多的信息？
• Clip256需要基于 Syncnet和 Frame256从头开始训练

不知道我的理解对不对，欢迎大家一起讨论哈

There are 5 models to be trained by DINet: Syncnet, Frame64, Frame128, Frame256 and Clip256. If you want to fine-tune a set of base models, which models need to be fine-tuned?

• Syncnet is relatively independent and can be fine-tuned directly based on the base model Syncnet
• Frame uses a coarse-to-fine training method. I think Frame64 can be fine-tuned directly based on the Frame256 base model. Later, I found that the net_g of Frame and Clip256 are the same. I feel that it may be better to fine-tune Frame64 based on the Clip256 base model. , so that more information in the base model can be retained?
• Clip256 needs to be trained from scratch based on Syncnet and Frame256

I don’t know if my understanding is correct, everyone is welcome to discuss it together.

[xiao-keeplearning]

针对专用人物的要嘴部更像的话，微调时可以基于 Clip256预训练模型调Frame64，128，256，最后clip256，这是可行的

[tailangjun]

针对专用人物的要嘴部更像的话，微调时可以基于 Clip256预训练模型调Frame64，128，256，最后clip256，这是可行的

好的，谢谢大佬