search

immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
52.02k stars 2.76k forks source link

[Feature]: Grouping of images based on faces found in the images. #294

Closed meet-minimalist closed 1 year ago

meet-minimalist commented 2 years ago

Feature detail

Google Photos and iPhone Photos have a feature where photos are organized based on faces found in the image.

Also, there are some self hosted solution providers which also have such feature.

  1. https://github.com/photoview/photoview
  2. https://github.com/LibrePhotos/librephotos
  3. https://github.com/photonixapp/photonix

It would be great if you can provide such feature in immich.

Platform

Mobile App

zackpollard commented 2 years ago

Hey, this is certainly something we have in the plans, however it's very difficult to do it in the way Google photos does it. We want to have this for the first official release of Immich but we want to do it right, so it might not happen for a little while as we investigate the best way to make this happen.

LackesLab commented 2 years ago

Hey there, I am a machine learning expert. I developed several applications which use facial recognition. I would like to support you with such a feature. I would also implement it by myself if appreciated? What’s the best way to contribute?

Update: I had a first look into code base and found out, that only tensorflow js is used. Due to the fact that the machine learning domain is settled in python, would it be possible to write a microservice in python providing the features as rest api?

PaulWoitaschek commented 2 years ago

Linking the corresponding PR: https://github.com/immich-app/immich/pull/641

xxyzz commented 1 year ago

I take a look of Photoprism's code and find they're using pigo to detect faces. Even though pigo is written in Go but it supports Webassembly, maybe it could also be used by Immich?

alextran1502 commented 1 year ago

@xxyzz , @LackesLab was planning to work on Face Recognition feature with state-of-the-art recognition methodology. We will take a look at this once we jump to this feature.

LackesLab commented 1 year ago

Hey all, sorry for not getting back to you in so long. I've had a lot on my plate and am still working on the feature. The creation of the face patterns is already done. I am currently still developing the way the faces are organized. I hope to find some time in the next weeks.

alextran1502 commented 1 year ago

Hey all, sorry for not getting back to you in so long. I've had a lot on my plate and am still working on the feature. The creation of the face patterns is already done. I am currently still developing the way the faces are organized. I hope to find some time in the next weeks.

No problem! Thank you for letting us know. Again, there is no rush on your part ☺️

Nonobis commented 1 year ago

Hello why not integrate https://github.com/exadel-inc/CompreFace in immich ?

samip5 commented 1 year ago

Adding search keywords: Regonize people from pictures

rhatguy commented 1 year ago

As you consider face tagging, please consider making the metadata readable and writable from the pictures and not just the DB inside Immich. There are many photo management solutions in place today and it seems like almost every one tries to use AI/ML to CREATE metadata, but they most simply store it in their database and not inside the EXIF information inside the photo. Hence when users move between photo management applications the metadata is lost. Given the time horizon people keep pictures (many many years) I would submit that keeping the metadata INSIDE the pictures (or in XMP sidecar) is critical. Think of how people write dates or names on the back of physical paper photos.

There seems to be a standard emerging in the industry for face metadata. Digikam is an example of a desktop app that is able to create/read/write/update face metadata. Other image galleries like PiGallery2 or MyLio are able to read and leverage that metadata written by Digikam. There is also a great tool for reading and writing the metadata in exiftool. Some people like myself have spent significant amounts of time tagging photos with faces, keyboards, geolocation and time, and tools that don't read that metadata and/or let us update that metadata going forward are a non-starter. For me, reading existing face metadata is more critical than creating new face metadata with AI/ML. There are other tools already that can create the face metadata (like Digikam). Any new metadata that is not stored with the picture will likely be lost when I inevitably try our a new photo application in the future.

MichaelBui commented 1 year ago

I totally agree with @rhatguy and I was about to comment the same thing. I hope Immich can live together with other DAM like DigiKam 👍

ark- commented 1 year ago

To extend to this, iOS photos also has face tagging and this data is stored in the EXIF as a collection of bounding boxes. However, the ID it assigns these boxes is unique to each photo (i.e. starts at 0, box 0 in photo X wont be the same face that is in box 0 in photo Y).

I assume Apple then has a separate lookup somewhere of Photo UUID + bounding box ID -> person.

2 possibilities:

  1. If there's a way to get at that lookup then importing faces and their grouping from Apple photos could be possible.
  2. This could save/enhance the "find a face bounding box" stage that Immich would need to do to implement facial grouping anyway. In my experience Apple's bounding boxes has been superior to other open source face detection DNNs. E.g. able to get people from the side in low light conditions

Just some thoughts.