D-Star-AI / dsRAG

High-performance retrieval engine for unstructured data
MIT License
852 stars 61 forks source link

Adding boolen in rse config to disable chunk length adjustment #55

Closed TaylorAndStubbs closed 1 month ago

TaylorAndStubbs commented 1 month ago

In use cases where documents range from chunk lengths of 40 - 700 characters it's better to simply disable adjusting the relevance scores via length. This keeps the defaults the same but allows a query to be made without the adjustment.