-
**Describe the bug**
Trying to use an Azure API Key to run a LLM evaluation using UpTrain. I received a 404 error message saying that the deployment is not found. However, there is no deployment name…
-
[ ] I checked the [documentation](https://docs.ragas.io/) and related resources and couldn't find an answer to my question.
**Your Question**
I wrote this code and I get the error:
The api_key …
-
[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.
**Describe the bug**
Error 'openai.APIConnectionError: Connection error.' occurs w…
-
**Is your feature request related to a problem? Please describe.**
I can't modify the existing template + rails on the evaluator object to customize for my use case.
**Describe the solution you'd…
-
Hey!
I have tried my hands on RAGAS with Watson LLM, the major issue I am facing is getting the warning:
"Failed to parse output. Returning None."
Continuously.
Is there any fix for this?
Is i…
-
This issue is now to track the implementation of various evaluation methods and workflows for LLMs.
Evaluations:
- [x] G-Eval
- [ ] PingPong
- [ ] InfiniteBench
- [ ] Ruler
- [ ] MMLU
- [ ] M…
-
Moving on to the **benchmark_models** tool in `evaluate_llm_tool.py,` and I had a few suggestions that might help improve it:
- Docstring Accuracy:
The docstring currently mentions a prompt_set…
-
I create a subclass of baseragassembeddings. because I already have all the embeddings for context, query, and question. I did this to not use the openai API key. because it is costly and also I want …
-
[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.
**Describe the bug**
I am trying to use a local LLM in the evaluate function., whe…
-
[ ] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.
**Describe the bug**
Error message is "Invalid n value (currently only n = 1 is su…