-
After the updates from https://github.com/Azure-Samples/ai-rag-chat-evaluator/pull/45 I've rerun the evaluation and got the minor score for all metrics:
Before update:
(I've omitted the context)…
-
I'm currently working on LLM evaluation using Trulens and Azure openai and encountering some issues with the code. Here's the relevant portion of my code:
```
retriever = vectorstore .as_retrieve…
-
Hi all,
Although `BIRD` has incurred significant annotation costs, we still cannot guarantee that all the data is accurately labeled. **_We hope that the community can assist us in building BIRD t…
-
### Issue you'd like to raise.
When i try to customize the LLM running the evaluation, i get the test to run without failling but it did not save the scores in Langsmith like it normaly does when i r…
-
Hello,
Is there a way to track progress when giving a list of inputs to a LLMChain object using tqdm for example?
I didn't see any parameter that would allow me to use tqdm.
I also checked if I c…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
Hello, thanks for your time in answering this!
I am running into some issues wit…
-
Hi:
Really interesting work on RAG evaluation metrics. However, I am unable to understand how these evaluation functions work. Could you please point me to some relevant documentation that explains…
vr25 updated
9 months ago
-
### Describe the bug
I am using Langfuse to monitor a LangChain AgentExecutor using AzureOpenAI model.
An error catch and logged by `langfuse.callback.CallbackHandler.on_chain_end`:
```
ERROR:langfu…
-
### This issue is for a: (mark with an `x`)
```
- [x ] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a beha…
-
console looked like:
```
python3 -m scripts evaluate --config=example_config.json --numquestions=14
```
error looked like
```
2024-01-13 00:04:04 (INFO) scripts: Running evaluation from c…