Closed shahules786 closed 4 months ago
Hi there! Thank you for your work!
I have a little question - is it possible to use this critic model for evaluation (not for test data generation)? I mean pass it as evaluate(llm=critic_llm), for example to calculate faithfulness.
If not, do you have plans to fine-tune model to replace gpt-4 in evaluation?
Hey @valer1435 the critic model is now fine-tuned only to act in test generation.
Why is it so hard to find a full working example of how to use a custom LLM for test set generation ?
I'm assuming this didn't help @projects-g -> https://docs.ragas.io/en/stable/howtos/customisations/ragas_custom_model.html
where are you stuck at?
Hi folks, please report issues regarding the ragas custom critic model in this thread.
Docs : https://docs.ragas.io/en/latest/howtos/customisations/ragas_custom_model.html Model: https://huggingface.co/explodinggradients/Ragas-critic-llm-Qwen1.5-GPTQ