Azure-Samples / ai-rag-chat-evaluator

Tools for evaluation of RAG Chat Apps using Azure AI Evaluate SDK and OpenAI
MIT License
162 stars 58 forks source link

Port to new promptflow-evals SDK #85

Closed pamelafox closed 1 month ago

pamelafox commented 1 month ago

Purpose

This PR ports the code to use the promptflow-evals SDK for the evaluation functionality, as the evaluate functionality is being deprecated in azure-ai-generative. The Q&A generation is still in azure-ai-generative for now.

Some user-facing changes:

Does this introduce a breaking change?

[X] Yes - see above. If you previously generated custom metrics, you'll want to rename keys in evalresults.json to the keys above.
[ ] No

Pull Request Type

What kind of change does this Pull Request introduce?

[ ] Bugfix
[X] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

How to Test