Open Xiyuche opened 4 weeks ago
I am also searching solutions like this, your idea is really nice! A customized Prompt should works fine according to #1121
I guess you don't need to modify Prompt in src/ragas/testset/prompts.py
but making a customized Prompt should work the same.
Your modifications are excellent, and I have implemented them accordingly. However, I encountered some issues during the evaluation. Could you please share the specific code you used for the evaluation?
Your modifications are excellent, and I have implemented them accordingly. However, I encountered some issues during the evaluation. Could you please share the specific code you used for the evaluation?
Sorry, what issues occurred? I didn't modify any other code other than src/ragas/testset/prompts.py
.
To be honest I'm currently using pip
installed ragas with english prompt, since it seems that the developing version is having more unexpected error.
您的修改非常好,我已经按照您的修改实施了。但是在评估过程中,我遇到了一些问题。您能分享一下评估时使用的具体代码吗?
抱歉,出现了什么问题?除了 之外,我没有修改任何其他代码
src/ragas/testset/prompts.py
。说实话,我目前正在使用
pip
带有英文提示的安装版 ragas,因为开发版本似乎出现了更多意外错误。
I followed your translation and found that it seems the prompt from prompts.py was not used; instead, only some prompts from the specific implementation of the metric were used.
您的修改非常好,我已经按照您的修改实施了。但是在评估过程中,我遇到了一些问题。您能分享一下评估时使用的具体代码吗?
抱歉,出现了什么问题?除了 之外,我没有修改任何其他代码
src/ragas/testset/prompts.py
。 说实话,我目前正在使用pip
带有英文提示的安装版 ragas,因为开发版本似乎出现了更多意外错误。I followed your translation and found that it seems the prompt from prompts.py was not used; instead, only some prompts from the specific implementation of the metric were used.
Oh yeah, I translated the prompt in prompts.py
because I want to do the Synthetic Test Data generation, not metrics or evaluation.
You can try to manually translate those prompts used in metrics or evaluation, that might works for you.
Thanks your idea. I try my Chinese text file it's worked. This is my example code https://github.com/zongzi531/modular-rag/blob/main/evaluation/ragas/generate_testset.py
Thanks your idea. I try my Chinese text file it's worked. This is my example code https://github.com/zongzi531/modular-rag/blob/main/evaluation/ragas/generate_testset.py
Thank you, this is very clever. I successfully ran your code, and it effectively avoids unexpected format issues when saving prompts to the cache.
Question Hi. I've tried language adaptation, it works totally fine the first time. But after that I received many different errors when the adaptor is trying to load the cached json files.
So I decide to use gpt-4o to translate all the english prompt content in
src/ragas/testset/prompts.py
to chinese , like this. I'm not very familiar with the project code, so I'm not sure about the effect, but the testset result seems to be fine.No need to worry about the chinese content if you cant read, I've manually checked the translated content, they are all good.
Code Examples