nextcloud / recognize

👁 👂 Smart media tagging for Nextcloud: recognizes faces, objects, landscapes, music genres
https://apps.nextcloud.com/apps/recognize
GNU Affero General Public License v3.0
519 stars 42 forks source link

Move to better face recognition models #1085

Open florianwgnr opened 5 months ago

florianwgnr commented 5 months ago

Describe the feature you'd like to request

I have installed Recognize together with Memories and so far everything works: Recognize scans my images for faces and groups them together. Great!

Yesterday, just for testing, i set up an instance of immich and loaded the same photo library i have stored in my Nextcloud. Immich currently uses the buffalo_l model from InsightFace for facial recognition, and after scanning i was blown away by the accuracy: The model not only found way more pictures, it was also way better at assigning them to the right person. To give you some numbers: recognize found 706 pictures of me, immich with buffalo_l found 3780 pictures. All correctly assigned with sometimes mind-blowing accuracy.

Since the results were that good, i thought about suggesting replacing of the face recognition model, e.g. with buffalo_l or similar, to further promote Nextcloud as the main storage for photos. I am aware that this is not a one-click replacement and might require a lot of effort or additional work, but as this might also affect ongoing discussions, e.g. https://github.com/nextcloud/recognize/discussions/865 regarding undetected faces, i wanted to share my thoughts with you.

Describe the solution you'd like

Move to an improved face recognition model

Describe alternatives you've considered

--

github-actions[bot] commented 5 months ago

Hello :wave:

Thank you for taking the time to open this issue with recognize. I know it's frustrating when software causes problems. You have made the right choice to come here and open an issue to make sure your problem gets looked at and if possible solved. I try to answer all issues and if possible fix all bugs here, but it sometimes takes a while until I get to it. Until then, please be patient. Note also that GitHub is a place where people meet to make software better together. Nobody here is under any obligation to help you, solve your problems or deliver on any expectations or demands you may have, but if enough people come together we can collaborate to make this software better. For everyone. Thus, if you can, you could also look at other issues to see whether you can help other people with your knowledge and experience. If you have coding experience it would also be awesome if you could step up to dive into the code and try to fix the odd bug yourself. Everyone will be thankful for extra helping hands! One last word: If you feel, at any point, like you need to vent, this is not the place for it; you can go to the forum, to twitter or somewhere else. But this is a technical issue tracker, so please make sure to focus on the tech and keep your opinions to yourself. (Also see our Code of Conduct. Really.)

I look forward to working with you on this issue Cheers :blue_heart:

bonswouar commented 3 months ago

A better face recognition model would really be needed, current one isn't doing so great imo. It seems face detection is done thanks to https://github.com/justadudewhohacks/face-api.js , and there haven't been any new commit there for the last 4 years..

I guess a PR on face-api.js with new models would be doable, but as far as I understand it would need to be available for tensorflow-js (see my comment on image detection models) @florianwgnr There are some tensorflow implementations of InsightFace that you quoted, but unfortunately they're all much older than the release of the model buffalo_l (septembre 2021 I believe)

tetebueno commented 2 months ago

How about this fork that's being mentioned in project's issues?: https://github.com/vladmandic/face-api

bonswouar commented 2 months ago

How about this fork that's being mentioned in project's issues?: https://github.com/vladmandic/face-api

It doesn't look like the models have been updated much

float3 commented 1 month ago

bumping this issue