How to train the model with confidence layer

[AaboutL]

Hi Marek, Thanks for your excellent work. I notice that you add a confidence layer in the FaceAlignment script, but it seems that there isn't a confidence layer in the FaceAlignmentTraining script. How do you manage the landmark loss and the confidence loss? How can I train the model with the confidence layer? Thanks.

[MarekKowalski]

Hi,

The training code for that layer has not been published so far. The additional layer is trained after the rest of the network has been pre-trained using the FaceAlignmentTraining script. Only the weights of this new layer are modified, while the rest of the weights remain frozen.

We trained the confidence layer using the WIDER dataset. To do so, we extracted a number of image patches containing faces and patches that do not contain faces from the dataset (each face image in the dataset is annotated).

Does that explain a bit? If you have more questions let me know, I will try to answer you quicker next time.

Marek

[AaboutL]

Hi Marek, Thanks for your kindly reply. I understand how you train the additional layer. But I still try to find a way to train them together. Do you have any suggestion? Is it a good way to give a weight between landmark loss and the confidence loss and add them up? Thanks.

[MarekKowalski]

Hi,

So just to make sure we understand each other: while we trained the additional layer separately, you would like to train it together with the rest of the network, is that correct?

You would definitely need some sort o a weight between the two errors. Moreover, since the datasets used to train the confidence layer and landmark loss are different (one has landmark annotations, the other doesn't), you would also need your loss function to be able to handle that fact.

Marek

[AaboutL]

Hi Marek, It seems a little harder to train the network together with the additional layer than train them separately. Thanks for your suggestion.

[Liz66666]

Hi Marek, How can I get the data to train confidence layer ? when I downloaded the WIDER dataset, how can I prepare the training data and labels? Thanks.

[MarekKowalski]

Hi,

The WIDER dataset is available here: http://mmlab.ie.cuhk.edu.hk/projects/WIDERFace/ As explained above the additional layer was trained separately, after all the previous layers were trained on the 300-W dataset. In order to train it we extracted a large number of image patches from the WIDER dataset. Some of the patches contained faces (we knew which, since the WIDER dataset has face annotations) and some did not.

During training the patches were fed to the neural network. The task of the confidence layer was to determine whether a given patch contains a face or not.

Best regards,

Marek

[stone226]

hi，Marek I want you give me more details about additional layer . 1 what is type the additional layer ?for example ,full connect layer or others. 2 How many layers the additional layer contains? just one or more than one. 3 where should i insert the additional layer in the network,if i train two stages landmark model Thanks.

[MarekKowalski]

Hi,

You can see the definition of the additional layer in line 117 in this file. It is only a single fully connected layer attached to the penultimate layer of the first stage.

Marek