multiple face problem

[abdou31]

I worked with your tool to annotated 300VW dataset with iris landmarks . When I execute FaceLandmarks.exe with this picture:

I get this output .csv : ( for multiple face ) :

I need just the coordinates of landmarks of the women who speaks. How can i know what landmarks are for the women who speak ? Is there a solution to detect only the landmarks of the women?

[TadasBaltrusaitis]

As in the 300VW dataset there sometimes are multiple people, there is no way for the approach to know which face is important or of interest to you. You have several ways to tackle this:

1. If you just use FeatureExtraction.exe, it will pick the biggest face, to start tracking from
2. You can filter out faces by size or closeness to the center of the image

[abdou31]

For now, i have filter this by adding in my script if condition by choosing face '0' , i think that face with id=0 is given the biggest face in the picture , did you agree with me?

[TadasBaltrusaitis]

It isn't guaranteed to be the biggest face, a safer way would be to either look at the scaling parameter (p_scale) or by computing a face bounding box.

[abdou31]

Can you explain more to me how can i do that?

[TadasBaltrusaitis]

You could just look at the column with p_scale header and pick the row that has the bigger value. This will give you the biggest face in the image.

[abdou31]

Thanks @TadasBaltrusaitis , i will try this after verify what causes bad results with my cnn ( i think 3D points causes the problem of bad prediction ) , i have just extract x and y coordinates (2D points ) from csv .

So i should correct three problems :

• Correct the prediction ( in 2D and in 3D position )
• Take the biggest face in the image
• Taking into account when iris landmarks when the eyes are closed ( landmarks should follow the action of the eyes ).

[abdou31]

When annotating the dataset using your tool and using pts_tools script that can draw points generated by openface , i get this result ( draw annotated points on the image ) :

I don't why the prediction with OpenFace is false with this image?

[TadasBaltrusaitis]

There will be frames where OpenFace provides a false prediction, especially on unseen datasets. As you mentioned one of the reasons could be because of variation in pose, combined with the microphone occluding part of the face.

[abdou31]

Do you know something? This dataset in the reality is a number of succssive frames produced by shell script from video , I think that it could be the reason that the prediction is give it to images instead of video My question is , what if i use FaceLandVid.exe instead of FaceLandImg.exe?

[TadasBaltrusaitis]

For video processing you should always be using FeatureExtraction.exe instead of FaceLandmarkImg.exe

[abdou31]

This is not video processing , those are images generated from videos