search

superlinked / VectorHub

VectorHub is a free, open-source learning website for people (software developers to senior ML architects) interested in adding vector retrieval to their ML stack.
https://superlinked.com/vectorhub/
Other
413 stars 97 forks source link

Movie Recommendation using VectorDB #385

Closed vipul-maheshwari closed 1 month ago

vipul-maheshwari commented 3 months ago

Article

svonava-superlinked commented 3 months ago

Hi @vipul-maheshwari thank you for contributing.

Before I dive into this, I noticed you say "production grade" in the summary - what for you makes this example qualify for "production grade"?

vipul-maheshwari commented 3 months ago

Hi @vipul-maheshwari thank you for contributing.

Before I dive into this, I noticed you say "production grade" in the summary - what for you makes this example qualify for "production grade"?

Hey Hi @svonava-superlinked , Well the retrieval and the similarity checks with the genre embeddings makes it more robust in terms of the recommendation! If you think I need to change this, let me know!

vipul-maheshwari commented 3 months ago

Thanks for the requested changes @morkapronczay , I am going to update the branch as per the requested changes and feedback. Pinging you soon

vipul-maheshwari commented 2 months ago

Changes to the Article Outline

Hello @morkapronczay,

I am making some changes to the article outline for a few reasons:

  1. Using embeddings from the hidden layer: Initially, we planned to use embeddings from the hidden layer for vector search. However, this approach seems too advanced given our basic neural network with only a couple of layers. The results may not be as effective as we hoped.

  2. New approach: Instead of using embeddings, I will use the classified genres. I have already implemented pre-filtering with LanceDB. Combining the classified genres and pre-filtering will help us provide better recommendations.

  3. Not using Movie overviews : Leaving overview part out of the training data as it turns out to be noise for the genre classification later.

Architectural Overview

Here is the updated architectural overview:

This new approach is more effective for our current setup.

vipul-maheshwari commented 1 month ago

Looking goooood! 🚀 Thanks @robertdhayanturner @morkapronczay , we did this together! More to come