(EAI-354): Discovery eval

mongodb / chatbot

MongoDB Chatbot Framework. Powered by MongoDB and Atlas Vector Search.

https://mongodb.github.io/chatbot/

Apache License 2.0

106 stars 48 forks source link

Open mongodben opened 3 days ago

mongodben commented 3 days ago

Add set of synthetically generated questions that don't mention mongodb to see if it's discovered in the answer.
Add regex-based conversation evaluator
Add discovery evaluation suite
Support concurrency on LLM conversation data generation
Support concurrency on evaluation stage