search

deepset-ai / haystack-tutorials

Here you can find all the Tutorials for Haystack 📓
https://haystack.deepset.ai/tutorials
Apache License 2.0
256 stars 87 forks source link

Problem with llm.meta in Tutorial: Model-Based Evaluation of RAG Pipelines #320

Closed greghobby closed 3 months ago

greghobby commented 5 months ago

Describe the bug I cannot run the Tutorial: Model-Based Evaluation of RAG Pipelines as written because when executed I get a runtime error.

Error message haystack.core.errors.PipelineConnectError: 'llm.meta does not exist. Output connections of llm are: replies (type List[str])

Expected behavior Much like llm.replies, I would expect llm.meta to be available for connect purposes

Additional context I have haystack-ai==2.0.1 pip installed

To Reproduce Run the code in its entirety

Run on Windows Enterprise 10

anakin87 commented 3 months ago

Now we have a new tutorial on evaluation -> https://haystack.deepset.ai/tutorials/35_evaluating_rag_pipelines

We also have 3 notebooks for integrations with other packages around evaluation: https://github.com/deepset-ai/haystack-cookbook/blob/main/notebooks/rag_eval_deep_eval.ipynb https://github.com/deepset-ai/haystack-cookbook/blob/main/notebooks/rag_eval_ragas.ipynb https://github.com/deepset-ai/haystack-cookbook/blob/main/notebooks/rag_eval_uptrain.ipynb

It's hard to tell which tutorial you were referring to.

I'm closing the issue. If the error persists, please reopen the issue and add some details.