[Feature-Request] Models for replacing Intel RealSense ID

[santicomp2014]

As of February 2022, Intel RealSense ID will be discontinued, I think OAK-D (IR VARIANT) with AI models could be a good replacement for F450/F455.

Neural nets for face recognition with AntiSpoofing, or other models for detecting a face and authorizing access. Not really sure it can be done yet without IR projectors, but this could be a good feature to implement

[Erol444]

Hello @santicomp2014! Do you know of any (opensource) models that do this? Otherwise it might be possible to take existing face recognition model (eg arcface) and modify it to take into account the disparity map as well. CC: @tersekmatija I'm not sure if the IR dot projector would be needed for this application, as face is a good "texture" for the disparity matching. But illumination might be useful for low-light environments. Also for such use-case a short baseline distance would be preferred.

[santicomp2014]

Hi @Erol444 , thanks for the feedback.

I'm not really aware of the specific models, but maybe arcface/deepface/mediapipe mesh could be used for this purpose. I'm not aware if these models have support for the stereo data, since they could be easily spoofed with a cell phone.

IR could be useful as you mentioned in low light scenarios, and also since the cellphone screen would appear black to the IR. It should take into account this stereo information to build a predictable model and recognize the user.

I looked at the firmware of F455 and I found these models in use, I did not find too much info on them.

[tersekmatija]

Hey @santicomp2014 , I think antispoofing could also be solved by adding another NN stage, for example something like this: https://github.com/minivision-ai/Silent-Face-Anti-Spoofing/blob/master/README_EN.md

Haven't tried this myself, but might be worth checking it out.

[santicomp2014]

I think maybe DeepFace + Silent-Face-Anti-Spoofing could be a good pair.

As the Author of Deepface mentioned

deepface comes with stream function and it is totally fine to use the library with stereo data.

I have to checkout the process of using both these models in parallel in the OAK-D. This is very interesting.

Any recommendations on using these custom models?. Can't wait to test it out on the OAK-D 😃

[tersekmatija]

Hey @santicomp2014 ,

I've used DeepFace a little bit before and it's awesome! This might make a good pair, actually, yes :)

To use the models on OAK-D devices, I would suggest a 2 NN stage pipeline (or possibly 3, as you have to detect faces as well - not sure if Silent-Face-Anti-Spoofing does this). So 1st stage would be face detector, then you could crop the image and pass it to Anti-Spoofing model. If crop would be recognized as real, you could then pass it to a DeepFace model. Not sure how fast this would be, but since you are focusing on good recognition, I think low FPS shouldn't be a problem?

For converting the models to blobs that can run on OAK-D, I would suggest this link: https://docs.luxonis.com/en/latest/pages/model_conversion/

Here is an example of 2 stage pipeline: https://github.com/luxonis/depthai-experiments/tree/master/gen2-age-gender

If you have any additional questions, feel free to post them here or on Discord! :)

[santicomp2014]

Thanks, @tersekmatija for the feedback, this is an Interesting pipeline. I still haven't received my OAK-D, but as soon as it arrives, I will test this configuration. Maybe the light version could also work with this configuration, or maybe the OAD-PRO would be interesting since it has IR.

What is the discord channel?.

Thanks again.

[Luxonis-Brandon]

We also have live-ish support on Discord, here: https://discord.gg/EPsZHkg9Nx

[tersekmatija]

Hey @santicomp2014 , I am pretty sure OAK-D Lite should also work with this configuration.

Once we release OAK-D-PRO, I'm pretty sure this will also provide some additional features that will be really useful in recognition pipelines :) Until then, I believe it's worth checking the discussed pipeline :)