Training files and parameters for the 4dface RCR model

[ardeal]

Hi

I ran your code on VS. I used 28 images(with 5 landmarks) for training. during test, if the test images are included in training images, the accuracy is very good. otherwise, the accuracy is very bad. I then used 3283 images to train the algorithms. The accuracy for some images are good, but some are not good.

I am wondering there is b0/bk mentioned in the author's paper, but I cannot find the implementation of b0/bk in you code.

Did you ever test your algorithms with more images? how about the performance? Could you please help to tell me about b0/bk question?

[patrikhuber]

Hi,

I don't think anything reasonable can come out from training with 28 images, I don't think that can generalise enough to work on test images.

I've trained with ~3300 images too (all ibug images), that was the training set for the 4dface model.

The accuracy for some images are good, but some are not good.

That's a bit hard to comment on without numbers - the model I trained had an average error similar to state of the art models.

I can't download the SDM paper right now but IIRC b0/bk must be the bias (or also called affine term). The code fully supports that, just add a row to the end of your matrix. I believe the rcr-train uses a bias term actually but I'm not 100% sure from the top of my head.

[ardeal]

Hi,

Many thanks for your answers!

I use the code landmark_detection.cpp to train the model with 3283 images, but the accuracy is really bad.

Maybe, I should use rcr-train. I will try this and tell you the latest output.

[patrikhuber]

Yes, definitely use rcr-train! The other app is just a simple demo. If you drop me an email, I can send you the parameters I used to train the 4dface landmark model.

On 8 Jul 2016 02:28, "ardeal" notifications@github.com wrote:

Hi,

Many thanks for your answers!

I use the code landmark_detection.cpp to train the model with 3283 images, but the accuracy is really bad.

Maybe, I should use rcr-train. I will try this and tell you the latest output.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/patrikhuber/superviseddescent/issues/28#issuecomment-231253498, or mute the thread https://github.com/notifications/unsubscribe/AEvLr_G5T97xpyEqEzGvCYBakZVLk75Rks5qTaeqgaJpZM4JG3TA .

[ardeal]

Hi,

Thank you so much for your quick response!

My email address: dpstill@126.com

I have one more questions: In your code, you push back all training image to a vector as Mat datatype. as we know, there are more than 3000 images. we need a very big memory for those pictures. how did you manage to do that? how much memory in your computer?

[patrikhuber]

I've sent you the files and parameters. Feel free to share this information, the only reason it's not on GitHub (yet) is I don't have time for it. If you have any questions, you can also ask them here publicly.

Regarding memory: I remember I made a change to store a reference-to-Mat, so each image will only be loaded once, and not for each perturbation, I think this should be live on GitHub. I believe the 3000 images required around 5GB of RAM but I could be mistaken, it's been a while - and our servers have >=64GB RAM ;-) Actually you can even calculate it by hand, they're 4-chan uchar images, you can assume an average resolution of 800x600 or so (Helen images might be larger?).

[Hardold]

Hi, I have the same problem as ardeal's. I use the rcr-train to train the model with ~3300 images(with 68 landmarks) which extract from video( http://ibug.doc.ic.ac.uk/resources/300-VW/), but the accuracy is not good. I trained the model according to the link(https://github.com/patrikhuber/superviseddescent/wiki/RCR-landmark-detection-training ). Could you give me some advice ?My email address: 2487625726@qq.com

[patrikhuber]

the accuracy is not good

That's not very specific. Did you just try in front of your webcam, or really test on a well-known database and evaluated the ROC / average error? One issue might be that there are no or few Chinese people in your training set, and then testing on Chinese people will naturally not give the best results.

I've uploaded the files I've used to train the 4dface model. You can have a look at them and the parameters. You need to remove the .txt extension.

• mean_ibug_lfpw_68_bbox_aligned.txt
• ibug_bboxes_mat2txt.m.txt
• rcr-train-from-bbox.cpp.txt
• rcr-test-from-bbox.cpp.txt

The parameters I used for one of the good models:

Set:
ibug-lfpw + ibug + afw + helen-tr
ground-truth bboxes

lms 68
perturbs 13
regressors 6
Regressor params (first to last):
MatrixNorm 5.0 false
MatrixNorm 5.0 false
MatrixNorm 5.0 false
MatrixNorm 5.0 false
MatrixNorm 5.0 false
MatrixNorm 5.0 false
HOG feature params for each regressor level:
VlHogVariantUoctti, 5, 11, 4, 1.5f
VlHogVariantUoctti, 5, 11, 4, 1.1f
VlHogVariantUoctti, 5, 10, 4, 0.7f
VlHogVariantUoctti, 5, 8, 4, 0.4f
VlHogVariantUoctti, 5, 6, 4, 0.25f
VlHogVariantUoctti, 5, 6, 4, 0.2f
perturb t_mu 0
t_sigma 0.08
s_mu 1
s_sigma 0.05
VerbosePartialPivLUSolver
InterEyeDistanceNormalisation

You can set them in the code. You can also modify some of them without incurring a large drop in performance (for example using only 4 regressors instead of 6 yields a model that is not too much worse).

[Hardold]

Hi, Thank you so much for your quick response and help !

[patrikhuber]

Renamed title so people can find the files and parameters in the future, or we can link to this issue.

Closing, feel free to reopen if it's regarding the same issue, or open a new issue otherwise.

[Sun1718]

Hi, Thanks for your amazing source codes.

I have a question about ".mat" file which is used for facebox creation. Which algorithm is used for creating of "bb_detector" from "bb_ground_truth"?

Best regards.

[patrikhuber]

Hi @Sun1718, not sure what .mat file you're talking about, but if it's a problem related to this library, please open a new issue, it doesn't seem too related to this one.

[Sun1718]

Thanks for your quick reply,

My question is about "ibug_bboxes_mat2txt.m.txt" file. ( As you mentioned above, you used this file for train the 4dface model). In this file, in the line: filename = '..\Bounding Boxes\bounding_boxes_helen_testset.mat'; you used "bounding_boxes_helen_testset.mat" file. In each cell of this file, there are three parameters: imageName , bb_detector and bb_ground_truth.

I don't know, how you calculate "bb_detector" parameter? calculate "bb_detector" from "bb_ground_truth"? how?

Thanks.

[patrikhuber]

I think the file bounding_boxes_helen_testset.mat is from the ibug website download page, is it not? If so, you should ask any questions pertaining to those zip files to the authors of them, the people from the ibug group.

You can also have a look at the code in this repo, how I read those files and what I do with its content - this should give you a good idea about what it is.

I think your question may even be answered on the page where you can download those zip files on the ibug website. I suggest reading that.

[Sun1718]

Oh, yes. you're right. I didn't know that bounding_boxes_helen_testset.mat is for ibug website.

Thanks for your help.