Hello! Thanks for providing such a good project.

chihfanhsu / gaze_correction

Correcting gaze by warping-based convolutional neural network in live video communication

283 stars 54 forks source link

Hello! Thanks for providing such a good project. #20

Closed ztrong-forever closed 2 years ago

ztrong-forever commented 2 years ago

What does the sum of absolute values of partial derivatives written in your paper mean?

ztrong-forever commented 2 years ago

sunnylulu commented 2 years ago

I am not the author, but I think the answer should be already in the paper. Please see the notes for figure 8.

We warp each eye image I to as similar as It to redirect the gaze. To retain the eyeball structure, pixels on pupil and iris should move similarly. We also expect pixels on eyelids to move similarly because of the low degree of freedom of eyelids.

That means the F should be same within iris.

chihfanhsu commented 2 years ago

Basically, it is the difference between two adjacency pixels along with the x- and y-axes. You can also calculate it by the 1D convolution with kernel [1 0 -1] such as https://en.wikipedia.org/wiki/Image_gradient?msclkid=89fdc03cbff011ec995750a27b5e2fef

Actually, you can find these in the code.