An issue about data processing

[sitadivon]

I got into trouble when I reproduced your paper, can you help me?

My question is about the processing of the Radbound dataset. I downloaded your script for your Radbound dataset and found that the face images were not equal to the number of landmark pictures.

I noticed what you mentioned in your paper: 'We first remove those images in who's face is not detected correctly the public OpenFace software, leading to 5,628 training information and leading 1,407 testing image pair'. The training set and the test set add up to 7,035 images, which corresponds to the number of face images which I downloaded (there are 1005 images in each face expression folder in the data set. if all the face images are combined with lankmarks, the total number of image pairs is just 7035).

But the problem is that you filtered the undetected landmark, so there aren't enough landmarks to pair with the face images. So I'd like to ask how much data you actually used in your experiments. If 7035 images were used, as the paper says, how do you solve the problem of the missing lankmarks?

[Ha0Tang]

In our experiments, each image pair is composed of a neutral and a non-neutral expressions. Thus the neutral expression images are used 7 times since we have 7 non-neutral expressions.

[sitadivon]

Thank you for your reply！

I totally understand what you mean. Maybe I didn't make it clear.

In the data set you provide, each face expression has 1005 images (including 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees) and can indeed lead to 1005 x 7 (7 non-neutral expressions)= 7035 pairs of images, which is consistent with the number mentioned in your paper (5628 images of training set + 1407 images of test set = 7035 images)

But in landmark folder, each face expression has only 603 landmarks (you dropped 0-degree and 180-degree view), so all 0-degree and 180-degree face images cannot be paired with landmark images.

For example, when I feed data to your C2GAN network, it requires 4 images which consist of A, B, landmark_A, landmark_B. But When I want to pair a 0-degree face image with 0-degree landmark image, I cannot find the 0-degree landmark image in the your dataset folder.

For myself, using the dataset you gave, I get up to 603 x 7 = 4221 pairs (which not equals to 7035 as your paper said).

I'm sorry to bother you again with these detail issues. I just want to figure out how many pairs of data you actually used in your experiments?

[sitadivon]

Here's the dataset information I downloaded with the script you provided: └── RaFD_image_landmark ├── RaFD_image (every face expression subfolder has 1005 images) │ ├── angry │ ├── contemptuous │ ├── disgusted │ ├── fearful │ ├── happy │ ├── neutral │ ├── sad │ └── surprised ├── RaFD_landmark_figure (every landmark subfolder has 603 images, they cannot form enough pairs with face folders) │ ├── angry │ ├── contemptuous │ ├── disgusted │ ├── fearful │ ├── happy │ ├── neutral │ ├── sad │ └── surprised └── RaFD_landmark_pts ├── angry ├── contemptuous ├── disgusted ├── fearful ├── happy ├── neutral ├── sad └── surprised

[Ha0Tang]

I will figure it out after ECCV ddl.

[sitadivon]

Please help me~! I need your answer to finish my graduation thesis. I will wait for you.

[Ha0Tang]

Hi, the 4221 pairs your downloaded are first split to 2814 training pairs and 1407 testing pairs. Then we do left-right flip on the 2814 training pairs for data augmentation as indicated in our paper, thus you will get 5628 training pairs. Finally, you will get 5628+1407=7035 pairs. Sorry for the late reply, hope this is helpful to you. If you have any further questions, please let me know. Thanks and good luck.

[Ha0Tang]

Please see the samples (with '_lr.jpg') on how to flip the images.

[sitadivon]

I get it! Thank you very much!