[FEATURE] Local privacy preserving face recognition (face identification)

[ivelin]

• Provide face recognition timeline events (in addition to face detection) for known faces
• Enable users to provide face recognition labeling by presenting them with detected faces and allowing them to provide tags or names for known faces. Similar to FB's friends tagging ,except no cloud image uploads.
• Use the labeling data to train a local on-device face recognition model.
• Allow multiple edge devices to collaborate in training via federated learning to better recognize people in a given household or professional facility premises.

Related resources:

• https://medium.com/@estebanuri/real-time-face-recognition-with-android-tensorflow-lite-14e9c6cc53a5
• https://pypi.org/project/face-recognition/
• https://medium.com/@BhashkarKunal/face-recognition-real-time-webcam-face-recognition-system-using-deep-learning-algorithm-and-98cf8254def7
• https://machinelearningmastery.com/how-to-develop-a-face-recognition-system-using-facenet-in-keras-and-an-svm-classifier/

Future work:

• Use federated learning to enable multiple edge devices (owned by the same user) to work together on recognizing user known faces without sharing image data with each other or any third parties.

[njcurtis3]

Does this local training method need to be trained using a pre-trained model on our own local machine using TF lite?

Also, I can take this assignment

[ivelin]

Great. Assigned to you now.

I am thinking that ideally this would be a trainable local model based on one of the open source pre-trained face identification models.

BTW, just renamed the title from face recognition to face identification. We already have a face recognition model built-in. It detects a face and pushes it to the timeline.

What we do not have is a model that can identify whose face it sees. In combination with UX improvement, we should be able to notify users when a specific family member is detected. Or when an unknown face is detected, give the user a chance to assign a family member identity label, discard it as not interesting, or mark as suspicious.

Over time we can integrate with third party sources such as registered sex offender public databases, which would alert users if a detected face matches the public police records.

[njcurtis3]

So, would the face identification be pulling from a user defined database? Meaning, will the user label their own/family members faces through the UI? Then, from then on the system would be able to identify who's face is being recognized? This is just an idea for a possible feature. Without thinking much through this, my initial intuition is that this would be beneficial for user control.

[ivelin]

Sounds like a reasonable way to go with the UX. An even lazier alternative may be to let the user tag a new face detection when it cannot be matched against any previously detected faces with known tags.

I think the facebook mobile app UX has been the most intuitive in my experience regarding face labeling. Not sure if that's true universally.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.