Visual search: Update grid with similar images using vector search

[akamil-etsy]

Description

is there a way to do something similar to https://same.energy/ by using typesense vector search and InstantSearch.js, i.e. when user clicks on an image update the hits grid with similar images retrieved via ANN. In other words image search from within the grid so users can refine the selection visually

i.m aware of typesense-instantsearch-semantic-search-demo but it has different functionality with click on Find Similar opening a separate window, and semantic search by text query rather than image.

[jasonbosco]

I'm actually working on a demo that is going to have a very similar "Pinterest-style" UI. But I wasn't planning to use Instantsearch.js for it, given that it wasn't designed to do this natively...

Any reason you want to use Instantsearch.js for it, vs building your own UI?

[akamil-etsy]

i dont have much experience with front end dev. looking forward for your pinterest style demo!

i think https://same.energy/ kind of UI would be ideal demo for Typesense, it has this 'wow factor' people like to share

[jasonbosco]

Ha! I just noticed a search bar on top of same.energy, which I wasn't planning on initially. I'll try to see if I can do this with instantsearch.js first.

My dataset is going to be HN comments, so all text. But the concept is the same - take some data (in my case HN comments, in same.energy's case - images), generate embeddings for it, index it in Typesense and then do an ANN search as each result (text / image) is clicked.

[akamil-etsy]

yeah, starting from text search and then refining visually is the most natural interaction it seems. The other option btw is starting from image and add text tags to refine

[jasonbosco]

I do like the idea of starting from text search and then drilling down from there visually.

[akamil-etsy]

btw same.energy has text demo for tweets as well https://t.same.energy/ (blog post)

[jasonbosco]

Oh interesting! Thank you for sharing