search

opensearch-project / neural-search

Plugin that adds dense neural retrieval into the OpenSearch ecosytem
Apache License 2.0
56 stars 58 forks source link

[META] Chunking and querying of long passages for vector search #612

Open samuel-oci opened 4 months ago

samuel-oci commented 4 months ago

Is your feature request related to a problem?

This is meta issue that orchestrate all tasks related to chunking and search of long passages and include both ingest, storage and query paths.

RFC ingest path: https://github.com/opensearch-project/neural-search/issues/548

RFC query path: TBD

RFC storage path: TBD

Issues: https://github.com/opensearch-project/k-NN/issues/1065 https://github.com/opensearch-project/neural-search/issues/482

yuye-aws commented 4 months ago

Thanks @samuel-oci ! This is the draft PR to implement fixed token length algorithm and ingest processor: https://github.com/opensearch-project/neural-search/pull/607.