slavyolov
commented
3 days ago Hints :
- https://medium.com/discovery-at-nesta/how-to-evaluate-large-language-model-chatbots-experimenting-with-streamlit-and-prodigy-c82db9f7f8d9
- https://www.databricks.com/blog/LLM-auto-eval-best-practices-RAG
- https://techcommunity.microsoft.com/t5/ai-ai-platform-blog/how-do-i-evaluate-my-llm-chatbot/ba-p/4139273
- https://www.iguazio.com/blog/llm-validation-and-evaluation/
Create evaluation dataset to validate the respones of the chatbot. Ideally this should be a saniteze ddsatasets with known answers (50/100) QAs.