search

AliaksandrSiarohin / first-order-model

This repository contains the source code for the paper First Order Motion Model for Image Animation
https://aliaksandrsiarohin.github.io/first-order-model-website/
MIT License
14.3k stars 3.18k forks source link

How to train the kp_detector model only? #454

Open GesilaA opened 2 years ago

GesilaA commented 2 years ago

The architecture of kp_detector defined in 'modules/keypoint_detector.py', and applied in 'GeneratorFullModel'. Loss function contains 'perceptual', 'generator_gan', 'feature_matching', 'equivariance_value', and 'equivariance_jacobian'. Which loss are designed for the kp_detector training？How to separate the keypoint model from 'GeneratorFullModel'? I'm looking forward for your replies.

AliaksandrSiarohin commented 2 years ago

You can not separate kpdetector from the rest. The most important aspect is the fact that network is able to reconstruct the driving image from the driving keypoints, so perceptual loss is the most important. However you can not separate it from generator training.

GesilaA commented 2 years ago

@AliaksandrSiarohin Thank you for your rapid reply! Is 'perceptual' loss used to learn the unsupervised keypoints, 'equivariance_value' and 'equivariance_jacobian' losses are designed for keypoints stabilizing? Or all of the losses would affect the keypoints learning?

AliaksandrSiarohin commented 2 years ago

All of them affect keypoint training.