REDBOX-453. Our context window gets a lot of junk that can confuse the LLM. Search scores often follow a distribution with a plateau and a drop – the “elbow”. This ticket is to design a method to find that elbow and cut results there.
Changes proposed in this pull request
Adds kneed to detect the elbow and truncate there
Adds a unit test to check it works
Adds scores to the objects retrieved from Elastic
Guidance to review
Check you're happy with my modified version of @lmwilkigov 's methodology
Check you're happy with a k=30, and elbow_filter_enabled=True as what we put into dev/preprod etc
Things to check
[x] I have added any new ENV vars in all deployed environments
Context
REDBOX-453. Our context window gets a lot of junk that can confuse the LLM. Search scores often follow a distribution with a plateau and a drop – the “elbow”. This ticket is to design a method to find that elbow and cut results there.
Changes proposed in this pull request
kneedto detect the elbow and truncate thereGuidance to review
k=30, andelbow_filter_enabled=Trueas what we put into dev/preprod etcThings to check