Questions about location of landmarks, normalised frame and bounding box

[cat3333]

Hi, thanks for this great job and answering my last question.

I have modified the 'static_input' example using Jointly learned multi-view models.

After running below codes, "printLandmarks(landmarks,flandmark->getLandmarksCount());" and "printLandmarks(flandmark->getLandmarksNF(), flandmark->getLandmarksCount());", there are 2 sets of numbers show in terminal (below is the screen capture of the 2 set of numbers).

Questions that I would like to ask are as follows. (1)Which set of number representing the location of landmarks, i.e. x and y coordinate of a landmark? (2)Are the landmark locations based on a normalised frame? (3)In 1 set of number, there are 2 rows(each row located between "[" and "]"), are the first number in each row represent the location of S01 and so on according to below picture captured from your paper?

(4)How can I get the normalised frame and draw the bounding box of the normalised frame just like the picture below?

(5) How can I get H and W? Are they equal to faces[i].x and faces[i].y or other?

I am sorry for asking so many questions.

[uricamic]

Hi @cat3333,

I am glad you asked, since now if anyone has the similar question, I can refer to this answer :)

1)The first set are landmarks coordinates in the original image, while the second set represents the landmark coordinates in the normalized frame. The format is s_i = [x_i; y_i], i.e. the first row are x-coordinates of landmarks, second row are y-coordinates and each column represents a landmark, they are numbered as in the images in the paper.

2) Yes, the landmarks are actually located just in the normalized frame. The coordinates in the original image are computed by back projection (based on coefficients of the affine transform computed during the normalized frame acquisition, i.e. the in-plane rotation + scale to a fixed size + translation of the origin).

3) see answer to 1). The only problem is the re-numbering of landmarks due to missing self-occluded ones. However, it is easy to compute the order based on the frontal view. In MATLAB it is as follows:

visible_subset = {
  [1 2 3       7 8 9       12 13 14       17 18 19    21] % -profile
  [1 2 3 4 5 6 7 8 9 10 11 12 13 14       17 18 19 20 21] % -half-profile
  [1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21] %  frontal
  [1 2 3 4 5 6 7 8 9 10 11       14 15 16 17 18 19 20 21] %  half-profile
  [      4 5 6     9 10 11       14 15 16    18 19 20 21] %  profile
};

views = {'-profile', '-half-profile', 'frontal', 'half-profile', 'profile'};

I will soon upload the learning scripts for the jointmv detector, where this problem is addressed.

4) The bounding box is easy to draw, since you need its coordinates anyway. To get the enlarged bounding box which is actually used to form the normalized frame, you can either use the facebox coordinates and bw_margin of clandmark class, or you can play with the getHinv() and getH() methods, which return the affine transformation matrix to get points to the normalized frame or back to the original image.

5) H and W are actually fixed in the model, parameter bw, use getBaseWindowSize() method to get the width and height of the normalized frame.

[cat3333]

Thanks for your quick reply and detailed solutions!!

They help me a lot! =)