search

gradio-app / gradio

Build and share delightful machine learning apps, all in Python. 🌟 Star to support our work!
http://www.gradio.app
Apache License 2.0
32.49k stars 2.44k forks source link

Add interactive plots within gr.Chatbot #4355

Open Gabriel-Macias opened 1 year ago

Gabriel-Macias commented 1 year ago

Is your feature request related to a problem? Please describe.
No.

Describe the solution you'd like
I'm currently looking into building a chat application that is able to show certain plots to users based on their requests and was wondering if is possible to embed other gradio components such as gr.Plot or some way to include interactive plots made via plotly or bokeh within the chatbox as opposed to including a separate gr.Plot component to display them outside of the chat. My current solution only considers inserting static images in the chatbot via hyperlinks.

abidlabs commented 1 year ago

Hi @Gabriel-Macias this is an interesting idea, but probably not one we'll support in the core library soon as it would require a significant refactoring of gr.Chatbot. I do see the value of it down the line, so let's keep this issue open

trivikramak commented 1 year ago

Hi @abidlabs, Is there any update on the timeline or priority of this feature/enhancement?

abidlabs commented 1 year ago

Not yet -- I think this will require quite a bit of work unfortunately. cc @dawoodkhan82

codenprogressive commented 8 months ago

Hi @abidlabs any chance this feature could be implemented? I see a lot of value having it part of Chatbot capabilities!

freddyaboulton commented 8 months ago

I think this could be an interesting custom component!

andreped commented 7 months ago

This would be of interest to me as well. We are able to do this in Streamlit without any additional libraries.

Does anyone know of a custom gr.Chatbot widget, which offers support for adding "any" gradio component such as gr.Dataframe or gr.Code inside a chat bubble?

codenprogressive commented 6 months ago

@freddyaboulton any snippet code you can share on how we can integrate plotly graphs in gr.Chatbot?

freddyaboulton commented 6 months ago

Hi @liwalayouni ! I'm not sure what the general solution looks like because there is no standard format for an LLM to specify a plot. For the plot case, I think you can handle this in the prediction function.

Parse the LLM output for a plot, save it to an image and then embed the image using html. Something like this <img src='/file={image}' /> where image is the image path.

codenprogressive commented 6 months ago

Hey @freddyaboulton assuming the LLM generated the chart and we have a Ploly Figure Object. The ask was how to plot it as interactive in the gr.chatbot. i.e can gr.Chatbot integrates a subcomponent like gr.Plot?

freddyaboulton commented 6 months ago

No built-in support. For now, I think you need to embed as an image or if you want interactivity, saving to html and embedding as an iframe in the response may work (have not tried)

https://plotly.com/python/interactive-html-export/