search

vaticle / typedb

TypeDB: the polymorphic database powered by types
https://typedb.com
Mozilla Public License 2.0
3.73k stars 338 forks source link

Vector Embedding Search Index #6911

Open sjpritchard opened 9 months ago

sjpritchard commented 9 months ago

Problem to Solve

I would like to be able to use vector embeddings to perform semantic searching over a TypeDB database, to support use cases like Retrieval Augmented Generation and integration with frameworks such as LlamaIndex. Vector embeddings are now increasingly supported across various database platforms, including Neo4j: https://neo4j.com/docs/cypher-manual/current/indexes-for-vector-search/

Current Workaround

None

Proposed Solution

Solution similar to Neo4j: https://neo4j.com/docs/cypher-manual/current/indexes-for-vector-search/

Additional Information

None

flyingsilverfin commented 9 months ago

Thank you for the request and the link to neo's resources! The approach is surely interesting and we should consider it - we haven't built much in the direction of advanced indexing (via Lucene for example) but it is something under consideration. Probably for our next roadmap it's not in scope, but when we develop the following one we can re-visit it.

sjpritchard commented 6 months ago

Just re-emphasising the importance of this. Graph databases are having a big resurgence due to AI and Retrieval Augmented Generation, and TypeDB is nowhere to be seen in the ecosystem. See https://python.langchain.com/docs/use_cases/graph/ and https://docs.llamaindex.ai/en/stable/examples/index_structs/knowledge_graph/KnowledgeGraphDemo.html#

haikalpribadi commented 6 months ago

Indeed it is something important for us to bring into TypeDB, @sjpritchard ! We'll look into adding it to our roadmap in 2024.