chroma-core / chroma

the AI-native open-source embedding database
https://www.trychroma.com/
Apache License 2.0
15.36k stars 1.29k forks source link

Add BM25 Full Text Search algorithm for hybrid search ability to Chromadb #1686

Closed HOAZ2 closed 1 month ago

HOAZ2 commented 9 months ago

Describe the problem

Please add the ability of the full text search with algorithm like BM25 for hybrid search solutions specially in RAG solutions. Right now, many advanced RAG solutions are depended on hybrid search solutions and Chromdb is one of the most used vector databases used for semantic search applications.

Describe the proposed solution

It would be great if chroma API would support/expose full text search feature.

Alternatives considered

No response

Importance

i cannot use Chroma without it

Additional Information

No response

atroyn commented 6 months ago

Chroma already supports full text search using the where_document feature: https://docs.trychroma.com/guides#filtering-by-document-contents

This should be much better named so people can find it more easily. We are looking into solutions for bm25 and similar.

nnnnwinder commented 4 months ago

Strongly looking forward to chroma joining hybrid search!

erikmargaronis commented 2 months ago

Chroma already supports full text search using the where_document feature: https://docs.trychroma.com/guides#filtering-by-document-contents

This should be much better named so people can find it more easily. We are looking into solutions for bm25 and similar.

where_document filtering is NOT full text search! I would also love to see support for BM25! :)

jeffchuber commented 1 month ago

closing in favor of https://github.com/chroma-core/chroma/issues/1330 - thanks for requesting this!

derevyan commented 4 weeks ago

I found best solution we move all to -> qdrant