About the accuracy of D435 facial recognition

[pgswag]

---

Required Info
Camera Model	D435
Firmware Version	5.15.0.2
Operating System & Version	Win 10
Platform	PC
SDK Version	{ legacy / 2.<?>.<?> }
Language	{C/C#/labview/nodejs/opencv/pcl/python/unity }
Segment	others

Issue Description

Two weeks ago, I posted a question about whether the D435's accuracy is good enough for 3D facial recognition projects. I was told it could. But when I used realsense viewer to view, and switch to “3D”, I found that the point cloud can not show the concave and convex sense of the face, so it can do the use of three-dimensional face information for face verification?

[MartyG-RealSense]

Hi @pgswag RealSense cameras can perceive curved surfaces but for faces they will have difficulty seeing the side areas of the face such as the ears on either side (like the image below from a D435 that I captured in the 3D mode of the RealSense Viewer).

The side detail of the head is not required for facial verification purposes though as the important verification details are on the front of the face.

[pgswag]

When I look at the information on the front of the face from the viewer, it doesn't reflect well either. Especially if you switch to 3d mode, such as nose, eyes, mouth, these features of the face are not well distinguished. I would like to know if the effect does not reflect the features of the front face as well as the following figure, for example, as the current D435 performance, is enough to support the use of facial 3D information for authentication?

[MartyG-RealSense]

Typically when using facial detection with depth information, the facial landmarks are matched up to an RGB image, like in the RealSense SDK's dlib C++ example program at the link below.

https://github.com/IntelRealSense/librealsense/tree/master/wrappers/dlib/face

So as long as depth can be clearly and accurately read from the face then it should not matter if the depth image does not resemble the one that you provided above.

[pgswag]

Does this face example project realize the recognition of faces or can it realize the authentication of different face information? For example, when A's face is close, it can match the database and tell it is A, and when B's face is close, the system says it is B. I want to know what kind of facial recognition is this, is it the kind where we have a red frame around our face when we take a picture or is it the kind that I use as an example?

[MartyG-RealSense]

The purpose of the Dlib example is simply to detect whether the face is a real one or a fake one (a flat photograph) based upon the facial landmarks detected. It cannot identify whether it is a specific person (Person A or Person B) because it is not connected to a database of faces.

The RealSense SDK 2.0 (librealsense) does not have support for storing individual faces in a database so that different faces can be identified.

---

Intel released a RealSense product called F455 that is a dedicated facial recognition solution with its own face SDK. It stores face data locally inside the camera hardware. The product was retired though so it is now only available for purchase whilst remaining stocks last.

https://store.intelrealsense.com/buy-intel-realsense-id-solution-f455-for-facial-authentication.html

https://github.com/IntelRealSense/RealSenseID

[pgswag]

What if I make my own database? I would like to know whether the accuracy of D435, that is, the capture of three-dimensional information about the face, is enough to identify whether it is a specific person (Person A or Person B) ?

[MartyG-RealSense]

It certainly should be accurate enough. Even the previous generation of less advanced RealSense cameras before the 400 Series (R200 and SR300) could capture facial data into a database and distinguish between individual faces when the person walked in front of the camera as the SDK for those cameras had built-in support for face functions. That old SDK does not support modern RealSense cameras such as D435 though unfortunately.

https://www.intel.com/content/dam/develop/external/us/en/documents/face-tracking-843462.pdf

[pgswag]

Thanks for your help,！！！！my problem is solved！