search

svthalia / concrexit

Thalia Website built on Django.
https://thalia.nu
Other
22 stars 12 forks source link

Add face recognition to photos #986

Closed thaliawww-service closed 4 years ago

thaliawww-service commented 4 years ago

In GitLab by @se-bastiaan on Dec 29, 2019, 12:33

One-sentence description

Add face recognition to photos

Motivation

Because it is really nice to be able to easily see all the photos taken of yourself.

Desired functionality

I think there are two ways to implement this:

  1. A single 'album' that shows all the pictures taken of you in reverse chronological order (oldest last).
  2. An option to only show photos of you in every photo album

My preference would be option 1. This is what I implemented as proof-of-concept.

Suggested implementation

Doing face recognition without GPU is really easy when using the face_recognition library. It is written in Python, has good documentation and an easy API. We would only need to change our docker images to include dlib somehow. An example of a Dockerfile to do this is in the PoC.

You might already have wondered how to do this without having to do an expensive operation every time you want to obtain the photos with your face.

In my PoC I indexed all faces on the 'large' image size using the API, which is a bit slow. But that is to be expected when you're loading images from a remote server. We can do the same, save the face encodings in a database and connect them to the photos they belong to. Then a user can upload images (no real need to save those, but it is possible) from which we will extract face encodings which we can compare using a SQL query to get similar faces. That should be a fast way to compare hundreds of encodings and option 1 is done!

Getting the face encodings from uploaded images is possible either when we're resizing and uploading or we can run a cronjob of some sorts to get those encodings.

If we really want to make this fancy we could include the face boxes (location of faces) in the images and add a possibility to tag people. That would assign the face encodings we extracted. But that's not ~"easy and fun" anymore probably.

thaliawww-service commented 4 years ago

In GitLab by @mtsatsev on Jan 29, 2020, 18:39

BEST THING EVER. Although we already discussed it and Neural network is not the way you decided to do it. GL still My suggestion will still be posted here:

  1. Add a face recognition to the app and generate data from there
  2. Train a neural network to recognize that person by the name he/she has in their account.
  3. Use it to search the photos.
thaliawww-service commented 4 years ago

In GitLab by @joren485 on Feb 20, 2020, 21:19

Although I think this is very cool, I am against adding this to the website.

For me this ticks all the boxes of feature creep:

In my opinion, this is a cool thing we could do for the lustrum (or something similar).

thaliawww-service commented 4 years ago

In GitLab by @se-bastiaan on Feb 21, 2020, 14:34

I don't actually believe that this is hard to maintain due to the fact that the only thing we add is code to detect faces and code to retrieve images based on the faces.

However, I do agree that it is not necessary and can exist perfectly outside of concrexit (mostly because it already does in the PoC).

It would be nice to be able to do this for the lustrum, that is a good idea. If we run a separate instance with a more refined version of the PoC that has styling and better caching of the API responses I think that would work. And we would add something as well. It would be nice if everyone who ever had an account would be able to do this as well if it's for the lustrum.

se-bastiaan commented 4 years ago

We're probably doing this for the lustrum so it is no longer concrexit related.