Closed yuyaya-foifoi closed 3 years ago
Hi, @yuyaya-foifoi
Thank you for your quick response. @hysts
About 1 and 3 thank you I got it. About 2, in my understanding, gaze_vector was a denormalized vector that is identical to 3D coordinates in the camera coordinate system. So my understanding might be incorrect?
@yuyaya-foifoi
Ah, sorry. My use of the word "(de)normalize" seems to have confused you. It means to normalize the head pose, not to make the gaze vector into a unit vector. About the normalization, take a look at the section 4.2 of the MPIIGaze paper.
@hysts
Thank you, my understanding is that normalization is necessary to handle different camera/person images.
Let me check to make sure that my understanding so far is correct.
Is the output here is the coordinates of the camera coordinate system, as described in the last sentence of 3.2 in this paper?
Scaling gaze_vector by multiplying a constant is just for visualization reason.
I'm sorry, but I have an additional question.
The distance from the origin in the camera coordinate system to the reference point (center of the face) is set here for calculating the scaling matrix, but am I correct that this is an assumption of the distance between the camera and a reference point? (Which does not necessarily apply to other cases.)
Can this be generalized? For example, it won't be applied for 2 by 2 rotation matrices and vectors which has a shape (2, 1).
@yuyaya-foifoi
Basically, yes. A slight difference is that in the last sentence of section 3.2 of the paper, M-1 is multiplied, but here S is ignored and only R-1 is multiplied because scaling a vector doesn't change its orientation.
Yes, as long as this line is concerned.
You're right. It's the ds in section 3.2 of this paper. Since it's used to normalize face images, it should be the same value as the one used to create the training data.
Yes, it can be generalized to any dimension. What do you mean it won't be applied to 2D cases? I'm confused. If A is an orthogonal matrix, AT = A-1. So (vT A)T = AT v = A-1 v, where v is a column vector. As a 2D rotation matrix is orthognal, it holds in 2D cases too.
Thank you so much for your quick reply. @hysts
If S is ignored, that means the absolute distance between the origin of the normalized camera coordinate system and the reference point will no longer be fixed?
Thank you I understood.
Hmm, I see... That's not a small constraint. Do you have any idea to remove this constraint? If I multiply by M during training and ignore S during inference, then will end up with data drift so...
Thank you I understood the transformation you shared with me. Simply I thought you are asserting holds. I might have missed transpose somewhere?
@yuyaya-foifoi
1 & 3. There seems to be some misunderstanding. The same preprocess should be applied to images during training and inference, and indeed the code does so. The difference I mentioned is not part of the preprocessing, but it's a post-processing after the model has predicted the gaze vector in the normalized head coordinate system. The length of the gaze vector means nothing, so we just ignore S in the post-processing.
@hysts
Thank you for explaining it so well. My questions have been cleared up.
Again, thank you for sharing your great work. I have a few questions. It would be a great pleasure if you reply to me.
Thank you,