search

melih-unsal / DemoGPT

🤖 Everything you need to create an LLM Agent—tools, prompts, frameworks, and models—all in one place.
MIT License
1.69k stars 200 forks source link

Trubrics Integration into DemoGPT #23

Open PBozmarov opened 1 year ago

PBozmarov commented 1 year ago

Hey @melih-unsal, this PR enables you to gather and analyze feedback for the results and sources of the DemoGPT model by integrating the Trubrics SDK. You are then also able to analyse the feedback received through registration on the Trubrics platform.

Setup

To store feedback in your Trubrics account, follow these steps:

  1. Register for a free account with Trubrics here
  2. Input your Trubrics email & password into st.secrets

Preview

Here is an example of gathering and storing feedback using the faces feature for the generated code from DemoGPT.

Screenshot 2023-07-24 at 16 42 19

We have also integrated feedback into the Streamlit app with the generated model:

Screenshot 2023-07-24 at 16 33 56

In the following image, we can see the feedback saved in our Trubrics account.

Screenshot 2023-07-24 at 16 41 42

Useful links

  1. See here for information on how to integrate Trubrics with Streamlit.
  2. Here you can find descriptions of the various feedback types (thumbs, faces, custom text).

We believe it would be fascinating to investigate the possibility of the feedback collector being automatically incorporated into the code when the model is generated, as opposed to being added externally as we did. Please let me know what you think.

If you have any questions, feel free to contact me.