About wrong tracking result of tracking-without-fd

[zjchuyp]

Hi patrikhuber, I build the tracking-without-fd, it usually has good tracking result. But sometimes, the tracking rectangle become small, and tracking result is wrong. I also test Intraface http://www.humansensing.cs.cmu.edu/intraface/download_functions_cpp.html, its tracking results more robust. How can program find when superviseddescent has a wrong tracking? thanks a lot!

[patrikhuber]

Hi @zjchuyp,

I just added a small check, now the tracking will reinitialise when a face is smaller than 50 pixel. This should help a bit.

IntraFace do not disclose their source and tracking method. However, my guess is that they use something like the average feature vector (learned during training) and then, during runtime, see how "close" the detection is to that. So in essence a small classifier. It would be easy to add that to the RCR and I have it on my todo-list.

Cheers, Patrik

[zjchuyp]

hi Patrik, You are right. I ask the author of Intraface, he tell me "tracking is detection in every frame, also uses SDM. The classifier is built upon the features extracted from the predicted landmarks. If the score is low, we say the face is lost. Otherwise, the tracking is good." I just wonder why there are two trained model(detection model and tracking model) in the author's Intraface demo.

[patrikhuber]

Hi @zjchuyp,

I just wonder why there are two trained model(detection model and tracking model) in the author's Intraface demo.

That's most likely because the tracking model is trained with different initialisations and perturbations. You train a detection model so it plays nicely when initialised with the coarse face box from a face detector, but in a tracking scenario, you have other assumptions, for example that the subject's face only moves a few pixels from its previous location. We can train a model using this assumption.

Cheers, Patrik

[zjchuyp]

Hi Patrik, I read your code of rcr-train, I wonder if something are wrong in function align_mean(). This function just return the cv::Mat mean, rather scaling the landmarks mean to size of face box.

cv::Mat align_mean(cv::Mat mean, cv::Rect facebox, float scaling_x=1.0f, float scaling_y=1.0f, float translation_x=0.0f, float translation_y=0.0f) { using cv::Mat; // Initial estimate x_0: Center the mean face at the [-0.5, 0.5] x [-0.5, 0.5] square (assuming the face-box is that square) // More precise: Take the mean as it is (assume it is in a space [-0.5, 0.5] x [-0.5, 0.5]), and just place it in the face-box as // if the box is [-0.5, 0.5] x [-0.5, 0.5]. (i.e. the mean coordinates get upscaled) Mat aligned_mean = mean.clone(); Mat aligned_mean_x = aligned_mean.colRange(0, aligned_mean.cols / 2); Mat aligned_mean_y = aligned_mean.colRange(aligned_mean.cols / 2, aligned_mean.cols); aligned_mean_x = (aligned_mean_x_scaling_x + 0.5f + translation_x) * facebox.width + facebox.x; aligned_mean_y = (aligned_mean_y_scaling_y + 0.5f + translation_y) * facebox.height + facebox.y; return aligned_mean; }

[patrikhuber]

Hi @zjchuyp,

Could you elaborate on what part do you think could be wrong? Like for example highlight the code? (I also think your code formatting got a bit messed up above).

This function just return the cv::Mat mean, rather scaling the landmarks mean to size of face box.

In the code, you can see that we scale the mean to facebox.width and height, and add facebox.x and y. Why would you think that this does not scale the mean?

In case you're wondering why scaling_x=1.0f etc., these are optional parameters for an additional scaling, but that's not necessary, as the mean is learned and correctly scaled during training.

Looking forward to hearing your comment.

[zjchuyp]

Hi Patrik, In the function , you just set "Mat aligned_mean = mean.clone();", after that, you have not set new values to "aligned_mean".

[patrikhuber]

@zjchuyp,

If you look at the following two lines:

Mat aligned_mean_x = aligned_mean.colRange(0, aligned_mean.cols / 2);
Mat aligned_mean_y = aligned_mean.colRange(aligned_mean.cols / 2, aligned_mean.cols);

This will not make a copy of the data - the variables on the left will point to aligned_mean, and in the following lines, it is modified.

[zjchuyp]

Hi Patrik,
I‘m sorrry, I don't know Mat share the same memory!

[patrikhuber]

I'm glad I was able to help!

Closing this, problem solved :-)