Novel search experiences like spatial / similar images

jasonjmcghee / rem

An open source approach to locally record and enable searching everything you view on your Mac.

https://rem.ing

MIT License

2.2k stars 63 forks source link

Novel search experiences like spatial / similar images #48

Open jasonjmcghee opened 5 months ago

jasonjmcghee commented 5 months ago

Lots of cool stuff that we can do here...

jasonjmcghee commented 5 months ago

https://huggingface.co/unum-cloud/uform-vl-english

tleyden commented 5 months ago

So to clarify, do you mean adding a unified search that combines searching over both:

embeddings of OCR'd text
image embeddings

jasonjmcghee commented 5 months ago

Potentially, definitely. Maybe there’s specifically a text embedding and/or an image embedding and/or a combination of both (like the model mentioned).

Depends what works.

And the medium of search might be text and/or some sort of image similarity interface.