feat: integrating with llama_index and its evals from ragas

[jjmachan]

So I'm the co-maintainer of ragas an evaluation tool for LLM pipelines and a contributor to llama_index, langchain and a few ml tools. We have been seeing that there is a solid use case for tracing, debugging and visualizing tools for people building LLM apps. I found your PR on llama_index and found it intriguing and was wondering if there was any possibility of a collaboration with your project and Ragas.

would love to have a chat 🙂

[eturchenkov]

Hi @jjmachan) It would be great to improve hayloft functionality by evaluation features from ragas. For now I think about something like protocol so different LLM frameworks/tools wil be enable to visualize valuable stuff for developers. Let's discuss evaluation features I can implement in hayloft package.

[jjmachan]

honestly, the protocol is the key. Llamaindex already has something with callbacks which is neat. wandb also has the prompt and tracer concepts which are neat but if we can make something that is easy to use for devs to add into their systems and then make a solid UI experience for that. It would be valuable.

if you share your emailID I'd love to chat more - I'll keep this issue open for ragas integration in the meantime, what do you say?

[eturchenkov]

I consider LLM apps will require more convenient approaches to track and improve them. I texted you to your email, let's discuss features for next milestones.