How to use it in js langchain environment?

[jung-han]

[x] I checked the documentation and related resources and couldn't find an answer to my question.

Your Question Hello, ragas Team! I am very impressed to see that they are making really good tools. I am creating a service using RAG, and while looking for a good evaluation tool, I ended up here.

I am not an AI developer, so I am not familiar with Python and am creating an AI service based on JavaScript. (Seeing that most AI-related tools are provided in python, I am wondering if I should change the language. 🥲)

From what I've found, I can't think of a way to test the model in a Langchain environment using JavaScript. Can you suggest a way to use Ragas?

I'll wait for your reply. thanks!

[jung-han]

⁺

After testing a little more, it seems that you can use it by creating a testset in a langchain js and then doing a little additional configuration to test this document in a python-based ragas app.

If there is a method that can be used right away or if I am misunderstanding something, please let me know!

[mongodben]

Following on @jung-han, I think it would be great to have some support for Ragas from a JavaScript/Typescript environment.

I work on a production RAG chatbot and framework (https://github.com/mongodb/chatbot), and have been quite underwhelmed with the RAG evaluation tooling available in the JavaScript ecosystem.

The currently best available options to the best of my knowledge are:

1. https://ts.llamaindex.ai/modules/evaluation/
   • It has a few evaluations, but the feature set is quite limited compared to Ragas
2. Rolling your own something (based on a Python library like Ragas)
3. Invoke Python from JavaScript, either via HTTP server call or call a python process directly (which can be fairly janky)

A few ideas on how to give more support for Ragas in JS:

1. Create a JavaScript library that has the same functionality as the Python
   • this would be nice for us JS devs, but presumably a lot of work on your end
2. Provide a "first class" way to call Ragas Python library from JS, such as creating a Docker image with a HTTP server where the endpoints correspond to different Ragas methods. You could even provide a lightweight JS SDK to query the server.
   • this is a similar idea to what unstructured.io has done with its JavaScript client SDK being able to call the Python lib in a docker image.

[jung-han]

@mongodben Thank you for your opinion. There are also techniques that I learned for the first time! 👍 I also considered creating a JS library, but since I'm not AI Man, I don't think I fully understand the internal code and structure of ragas. Updating and maintenance seems difficult.

For now, I want to use ragas so I'm considering the second method you mentioned Since the RAG app that we want to evaluate does not receive and process data in real time, we also consider creating a separate pipeline that creates a dataset for evaluation in the JS app and then loads and evaluates the dataset in the ragas python app. Although it may be difficult to test each component provided by ragas, I have hopes that it will be possible to test e2e. (Come to think of it, with this level of evaluation, it would be okay to use llamaIndex.ts!)

[jung-han]

⁺ I am currently configuring an app using langchain. I tried using llamaindex for evaluation, but I ended up rewriting the code for RAG. 🥲

[linhnd34tt]

@jung-han I was looking for as well, have you tried this? https://js.langchain.com/v0.1/docs/guides/evaluation/string/criteria/