search

xemle / home-gallery

Self-hosted open-source web gallery to view your photos and videos featuring mobile-friendly, tagging and AI powered image discovery
https://home-gallery.org
MIT License
836 stars 64 forks source link

Object recognition has some limitations #152

Open fire-eggs opened 2 months ago

fire-eggs commented 2 months ago

image

image

image

image

On the other hand, sometimes works very well: image

Sometimes it's fun! image

xemle commented 2 months ago

Hi.

Object detection uses ML/AI and relies on work of giants. So the basic functionality is not possible to implement by your own and relies on heavy trained neural networks by others. The used ML model is cocoSsd with mobilenet v2 (see api-server/src/model-config.js.

The result is kind of 'take it or leave it' and the only setting is the probability score of a detected object. The min score is currently set to 0.6, see (database/src/media/objects.js)[https://github.com/xemle/home-gallery/blob/master/packages/database/src/media/objects.js#L26].

Maybe you play around with different min scores and check which value fits you best.

FYI Since v1.16.0 HomeGallery supports plugins and you can implement your own extractor and database mapper. With it you can maybe explore other ML solutions and integrate it to your gallery. Also other self hosted galleries might offer you a similar ML server like the ApiServer for HomeGallery - I have not checked it but the chances are quite high, since all the galleries needs to do the same thing regarding fancy ML stuff which is most probably dockerized. ... and if you find some promising solution, please share it ;-)