Closed mongodben closed 1 month ago
@ankrgyl back to you
Autoevals (main-1725908744) Score | Average | Improvements | Regressions |
---|---|---|---|
NumericDiff | 73.5% (0pp) | 5 🟢 | 7 🔴 |
Duration | 2.53s (+1s) | - | 100 🔴 |
Llm_duration | 1.96s | - | - |
Prompt_tokens | 279.25s (+0s) | - | - |
Completion_tokens | 16.9s (+0.04s) | 13 🟢 | 14 🔴 |
Total_tokens | 296.14s (+0.04s) | 13 🟢 | 14 🔴 |
Estimated_cost | 0$ | - | - |
Currently, the embedding model used by the Ragas
AnswerRelevancy
metric isn't configurable. It defaults to the default ofEmbeddingSimilarity
, which is currently"text-embedding-ada-002"
.This PR makes the model configurable.
This is useful for all users who don't want to use the model "text-embedding-ada-002" or for Azure OpenAI users who use that model but give it a different name (which is configurable in the Azure OpenAI service)