profile face training

[nyck33]

I am getting the iBug dataset that includes profile face annotations. Can I use your training script to train a detector on profile face landmarks?
Here is a link to the 39 points profile landmarks I am referring to: 39 point landmarks

[Luca96]

Hi, if the dataset is structured and annotated like the iBug 300W dataset (the one provided by Dlib, which I've used), having an xml annotation file (with 39 landmarks, bounding-box and path of the image), you can use the training_script.py.

In this case, you only need to do few modifications to the script, like: edit the set of landmark, dataset path, model name and, optionally, the training parameters. Otherwise, you have to create a properly structured xml file, and then launch the script.

If you want, you can share your results by contributing to the repo. I would appreciate a lot.

Hope it Helps.

[nyck33]

@Luca96 I don't mind doing this because I probably need a mobile version later. But do you think I should use a full-scale (large) dlib model if it's for a PC application and I need as much accuracy as possible?
What was the actual difference in accuracy from large model?

[Luca96]

Well, choosing among an heavy (but accurate) model versus a light (faster, but less accurate) model is task dependent.

I mean that by fine-tuning the training parameters you can trade-off among speed, size and accuracy according to your application requirements.

I'm not able to quantify the difference in accuracy of a large model vs a small one, but, from my experience, the difference exists and sometimes can be seen just by looking at the predictions.

Anyway, for real-time applications (both PC or mobile) you need low inference time. You can achieve this by decreasing the feature_pool_size parameter, but this also reduces a bit the accuracy. So, its a trade-off.

Of course, mobile users will apprecciate a lot light and fast applications (low donwload time and smooth user experience), so a model should be as small as possible but without sacrificing too much accuracy.

Instead, when accuracy is fundamental you want to trade speed and size for increasing it.

You can find a description of the training parameters here.

[nyck33]

@Luca96

I am downloading the ibug 300 dataset from the link you provided. I have another one called menpo_train_release and menpo_test_release that I had to email a request from my school email account. But the license says it's only for academic use so I am not sure I can upload a trained model here. Please clarify on this point. Would you happen to know how to make an xml file for dlib from .pts files? Noxml file is provided for the profile and near frontal datasets. I guess this: https://stackoverflow.com/questions/26056316/how-to-convert-txt-file-into-xml-file-using-python.

[Luca96]

Sincerely, I have no idea about the restrictions of an academic license. I think that you should carefully read the license document, because it can vary according to institutions.

I think that pts is a kind of proprietary format. So, you should read it as a text file, and try understand its format. Then, you can begin to create the xml file.

To create the xml you need 3 things: paths of images, face's bounding box (usually provided by Dlib HOG face detector), and landmarks. If the .pts file lacks some of this things, you have to provide for them, like: detecting faces (if no bounding-box are provided), determining paths, etc.

The last note is about the format of the xml file. If you have downloaded the ibug dataset, there are a couple of xml files (e.g. labels_ibug_300W_test.xml). You need to mimic that structuring for your own xml file.

[nyck33]

Hey Lucas,

Well apparently the dataset is available on Github (can post here if interested) but the license is academic. And the owner just pointed out the bounding box coordinates for me so I'm good to go mimicking the xml. I'll definitely send you the model via email or something and let you decide what you want to do with it. Thanks for the help.

---

From: Luca Anzalone notifications@github.com Sent: Wednesday, August 28, 2019 11:58 AM To: Luca96/dlib-minified-models dlib-minified-models@noreply.github.com Cc: Toyoda Kim, Nobutaka T nobutaka@gatech.edu; Author author@noreply.github.com Subject: Re: [Luca96/dlib-minified-models] profile face training (#3)

Sincerely, I have no idea about the restrictions of an academic license. I think that you should carefully read the license document, because it can vary according to institutions.

I think that pts is a kind of proprietary format. So, you should read it as a text file, and try understand its format. Then, you can begin to create the xml file.

To create the xml you need 3 things: paths of images, face's bounding box (usually provided by Dlib HOG face detector), and landmarks. If the .pts file lacks some of this things, you have to provide for them, like: detecting faces (if no bounding-box are provided), determining paths, etc.

The last note is about the format of the xml file. If you have downloaded the ibug dataset, there are a couple of xml files (e.g. labels_ibug_300W_test.xml). You need to mimic that structuring for your own xml file.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/Luca96/dlib-minified-models/issues/3?email_source=notifications&email_token=AGAFZKPLGRDPIVKWLRDOW5DQG2ODDA5CNFSM4IPIK2U2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5LTR6A#issuecomment-525809912, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGAFZKMEMYW7Y5JSCKSMYS3QG2ODDANCNFSM4IPIK2UQ.

[nyck33]

Here's the dataset: https://github.com/nyck33/MenpoBenchmark

---

From: Luca Anzalone notifications@github.com Sent: Wednesday, August 28, 2019 11:58 AM To: Luca96/dlib-minified-models dlib-minified-models@noreply.github.com Cc: Toyoda Kim, Nobutaka T nobutaka@gatech.edu; Author author@noreply.github.com Subject: Re: [Luca96/dlib-minified-models] profile face training (#3)

Sincerely, I have no idea about the restrictions of an academic license. I think that you should carefully read the license document, because it can vary according to institutions.

I think that pts is a kind of proprietary format. So, you should read it as a text file, and try understand its format. Then, you can begin to create the xml file.

To create the xml you need 3 things: paths of images, face's bounding box (usually provided by Dlib HOG face detector), and landmarks. If the .pts file lacks some of this things, you have to provide for them, like: detecting faces (if no bounding-box are provided), determining paths, etc.

The last note is about the format of the xml file. If you have downloaded the ibug dataset, there are a couple of xml files (e.g. labels_ibug_300W_test.xml). You need to mimic that structuring for your own xml file.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/Luca96/dlib-minified-models/issues/3?email_source=notifications&email_token=AGAFZKPLGRDPIVKWLRDOW5DQG2ODDA5CNFSM4IPIK2U2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5LTR6A#issuecomment-525809912, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGAFZKMEMYW7Y5JSCKSMYS3QG2ODDANCNFSM4IPIK2UQ.

[vinicius121]

Hi @Luca96 and @nyck33 ,

I'm curious to know if you (@nyck33) managed to train the a detector with the iBug dataset that includes profile face annotations. I'm also interested to train one. If you managed it, could you say the steps you did? For detecting the face I'm using OpenCV CNN face detector as I found to be the best. So @Luca96 , if I train a model from scratch do I have to update the face bounding boxes of the model? The trained models, like the Dlib's original one, assume that the face is located by the Dlib HOG face detector. And if you (@nyck33) have already a model trained is it possible to share (I'm only going to use for academic purposes)?

Thanks

[nyck33]

I did not. I'm too busy with other projects

Get Outlook for Androidhttps://aka.ms/ghei36

---

From: vinicius121 notifications@github.com Sent: Wednesday, November 27, 2019 11:40:22 AM To: Luca96/dlib-minified-models dlib-minified-models@noreply.github.com Cc: Toyoda Kim, Nobutaka T nobutaka@gatech.edu; Mention mention@noreply.github.com Subject: Re: [Luca96/dlib-minified-models] profile face training (#3)

Hi @nyck33https://github.com/nyck33 ,

I'm curious to know if you managed to train the a detector with the iBug dataset that includes profile face annotations. I'm also interested to train one. If you managed it, could you say the steps you did? And if you have already a model trained is it possible to share (I'm only going to use for academic purposes)?

Thanks

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/Luca96/dlib-minified-models/issues/3?email_source=notifications&email_token=AGAFZKK32ER7YOYX7ZJV2WTQV3EKNA5CNFSM4IPIK2U2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEFKR23I#issuecomment-559226221, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGAFZKNTVDKEK5QGQLJGMTDQV3EKNANCNFSM4IPIK2UQ.

[Luca96]

@vinicius121, If your dataset is already annotated with faces, you can reuse that bounding-boxes unless you note some performance-issues like shifted landmarks and similar.

In that case you want to substitute the dlib's hog-detected face bbox with your CNN face detector. Note that the faces detected with a different face detector (e.g. OpenCV CNN) could be different from the ones within the dataset, so you may not be able to use the corresponding face landmarks too.