What if existing content get updated and new contents created?

[limcheekin]

Thanks for sharing the code. What happen when the existing content get updated and new contents created, do it need to create embeddings for all contents again? The current approach is not good as create embeddings cost money? Please see https://github.com/mpaepper/content-chatbot/blob/main/create_embeddings.py#L49. Would it be possible progressively update the vector store? Please advise. Thank you.

[mpaepper]

Yes, it's possible, but not implemented, yet. Basically you need to have some kind of observer that on changes reindexes the changed part, creates a new embedding just for that part and updates the vector store for that part.

That way, cost will be minimal.

[limcheekin]

Thanks for quick response.

FAISS vector store that exported to pickle can be updated?

[mpaepper]

Yep, you can load the Faiss store, change it and the save it again.