Confidence precision levels in CSV file

[tsschwartz99]

When I have media that is passed through FeatureExtraction, the resulting CSV file's confidence column is always rounded to the hundredths place. I thought this may have been caused by setprecision(2) in the PostprocessOutputFile function in FaceAnalyser. However, when changing 2 to my desired level of significance, the CSV file did not change. In FeatureExtraction, the value of face_model.detection_certainty has more significant figures (only 3.. would still prefer more) that do round up to the value present in the CSV file.

Any ideas on where I can find where this rounding is occurring? Thank you.

[brmarkus]

Can you share an example (executable, command-line parameters, ideally medium/video/input, CSV output (snippet))?

[brmarkus]

Try this: https://github.com/TadasBaltrusaitis/OpenFace/blob/369a92caa7a8876be45e27fb660ba87f9ef81dc2/lib/local/Utilities/src/RecorderCSV.cpp#L192

[tsschwartz99]

@brmarkus Thanks for the quick response! Your suggested edit did change the amount of significant figures in the CSV file. Very, very helpful. Thank you again.

Do you have any idea of where face_model.detection_certainty is calculated? I'm hoping that I could also potentially increase the amount of significant figures there as well.

This is the command I am using: FeatureExtraction -f <video file> -q -pose -3Dfp -aus -pdmparams -2Dfp -out_dir <path to output dir>

[brmarkus]

I think its retrieved here: https://github.com/TadasBaltrusaitis/OpenFace/blob/3d4b5cf8d96138be42bed229447f36cbb09a5a29/lib/local/LandmarkDetector/src/LandmarkDetectorFunc.cpp#L451 and https://github.com/TadasBaltrusaitis/OpenFace/blob/3d4b5cf8d96138be42bed229447f36cbb09a5a29/lib/local/LandmarkDetector/src/LandmarkDetectorFunc.cpp#L453

But isn't this actually the value written into the CSV into the column confidence...?

EDIT: The visualizer gets and remembers it: https://github.com/TadasBaltrusaitis/OpenFace/blob/3d4b5cf8d96138be42bed229447f36cbb09a5a29/lib/local/Utilities/src/RecorderOpenFace.cpp#L470C8-L470C37

and then write into the CSV: https://github.com/TadasBaltrusaitis/OpenFace/blob/3d4b5cf8d96138be42bed229447f36cbb09a5a29/lib/local/Utilities/src/RecorderOpenFace.cpp#L339

[tsschwartz99]

@brmarkus I apologize for my late response.

Yes, detection_certainty is what is written to the CSV file. But looking at these code references in the most recent reply, it is still not clear where this value is actually calculated. Despite changing the level of significance as suggested in the above reply (output_file << std::setprecision(2);), the confidence in my resulting CSV files only go to the thousandths place, despite having my significance set to 5 decimal places. Most of the non-zero results end in a .005 as well. This seems odd and I wanted to see if there is any additional rounding happening in when calculating this confidence. I know there isn't any after output_file << std::setprecision(2);.

[brmarkus]

I think the magic happens in the multiple "neural networks". Unfortunately I'm not familiar with the used undelying network architectures - otherwise there would be a chance to look into those with e.g. the viewer "Netron", to analyse especially the outer, last layers.

You might want to have a look intho e.g.

• https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/intel/facial-landmarks-35-adas-0002
• https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/intel/facial-landmarks-98-detection-0001

to get an impression what OpenFace's models might do as well. Under the hood it might use FP32, FP16, or uses INT8 quantization or something like that.

Excerpt for "facial-landmarks-35-adas-0002":

You will see convolutions, softmax, dropout, weight-filter, batch-norm, flatten and many more.

[tsschwartz99]

Perfect, thank you @brmarkus! I think that addresses my questions.