search

AliaksandrSiarohin / first-order-model

This repository contains the source code for the paper First Order Motion Model for Image Animation
https://aliaksandrsiarohin.github.io/first-order-model-website/
MIT License
14.3k stars 3.18k forks source link

confuse about the equivariance_value #154

Open Erdos001 opened 4 years ago

Erdos001 commented 4 years ago

Hi @AliaksandrSiarohin, thanks for share the code, but i am confused about the equivariance_value implemented in model.py

        ## Value loss part
        if self.loss_weights['equivariance_value'] != 0:
            value = torch.abs(kp_driving['value'] - 
            transform.warp_coordinates(transformed_kp['value'])).mean()
            loss_values['equivariance_value'] = self.loss_weights['equivariance_value'] * value

I think the keypoint of transformed frame should be equal to the transformed keypoint of the origin:

       ###
        if self.loss_weights['equivariance_value'] != 0:
            value = torch.abs(transform.warp_coordinates(kp_driving['value']) - 
            transformed_kp['value']).mean()
            loss_values['equivariance_value'] = self.loss_weights['equivariance_value'] * value
AliaksandrSiarohin commented 4 years ago

Well this is complicated part. If you have a transform A, when applied to the keypoints it is applied forward A*kp. However when when you apply same transform to image using grid_warp, it is applied backward (I.e. Backward optical flow) A^-1.

Erdos001 commented 4 years ago

Well this is complicated part. If you have a transform A, when applied to the keypoints it is applied forward A*kp. However when when you apply same transform to image using grid_warp, it is applied backward (I.e. Backward optical flow) A^-1.

Thanks for your rapid reply, I have misunderstood the grid_warp before, thanks again!