search

opendatahub-io-contrib / genai-llm-rag-app

Apache License 2.0
3 stars 3 forks source link

Introduce user feedback via evaluation #2

Open caldeirav opened 4 months ago

caldeirav commented 4 months ago

As a model admin, I want to be able to collect the feedback on model responses and use it to direct model finetuning.

Consider and review suitable frameworks such as https://langfuse.com/ which provide standardized scoring and evalution of model responses. The content should be usable in a review process that would rely on instruction tuning (possibly using a low code approach such as InstructLab integrated in a User Interface)

tuhinsharma121 commented 4 months ago

anyone working on it?

caldeirav commented 4 months ago

This is new issue open to contribution, starting with the design approach to integrate this with the app that @avijra is contributing.