whyhow-ai / knowledge-table

Knowledge Table is an open-source package designed to simplify extracting and exploring structured data from unstructured documents.
MIT License
196 stars 24 forks source link

[FEATURE] Implement reranking for chunk retrieval #15

Open tomsmoker opened 1 week ago

tomsmoker commented 1 week ago

What

Implement a reranking, or reprioritisation, of the chunks returned from the input query.

Why

Reranking the chunks post retrieval relative to the query can greatly improve results.

Implementation guidance

Create a services/reranker.py file implementing the logic, and import it into the llm.py file within the generate_response function.