gradio-app / gradio

Build and share delightful machine learning apps, all in Python. 🌟 Star to support our work!
http://www.gradio.app
Apache License 2.0
32.23k stars 2.41k forks source link

add `ignoredClasses` parameter for Chatbot KaTeX render #4844

Closed Keldos-Li closed 1 year ago

Keldos-Li commented 1 year ago

Is your feature request related to a problem? Please describe.
In our project, part of the scene and output do not need to be rendered by KaTeX, and some are needed. However, once rendering is enabled, all output is rendered, and even if we escaped $ with $, KaTeX still rendered everything. That totally broke what we've made. I believe there will be others who have the same problem.

Describe the solution you'd like
Can you add ignoredClasses(https://katex.org/docs/autorender.html) option parameter for KaTeX render to Chatbot, just like you've added latex_delimiters? Therefore, we can pass <span class="no-latex">.....</span> or <div class="raw-message">...</div> or something else to chatbot, and they won't be rendered.

Additional context
We would really appreciate it if you can resolve this so that our output switching function can work properly!

image
abidlabs commented 1 year ago

Hi @Keldos-Li would it not be easier to just set the LaTeX delimeters to something else, like two dollar signs and only use that when you want to render LaTeX?

Keldos-Li commented 1 year ago

No, unfortunately we cannot do this. The functionality we want to achieve is to display both the raw output of AI and the output after rendering with Markdown (including code blocks and LaTeX rendering, etc.). Even if it is not necessary to display, we have provided a copy button in the UI to copy the raw output of the model for users to paste elsewhere.

Since switching to front-end rendering of Markdown output, we have completely rewritten this feature and resolved most of the issues. However, the rendering of KaTeX cannot be solved.

In fact, it is necessary to disable rendering for people who need to write formulas in LaTeX in certain situations(html tags). For example, if I need ChatGPT to help me write an assignment explaining a physical concept, I need to check if ChatGPT's output is correct in the rendered view. If it is correct, I can directly copy the raw model output and paste it into my LaTeX editor (or Markdown editor).

It is not because "$" will cause rendering errors in plain text, so setting LaTeX delimiters cannot solve our problem. (However, we have indeed adapted the LaTeX delimiter functionality for users to choose whether to always render LaTeX formulas or not.)

And now, we have completely lost the possibility of copying the raw output in LaTeX. When facing a rendered complex formula, selecting and copying the formula will only result in garbled text, and users cannot know how this complex formula is written.

For us, if we continue to use Gradio, adding ignoredClasses in the KaTeX parameters is the most convenient and effective solution to our problem...


Supplement:

What we get now:

image image

What we wanted to get:

image image

It's all about the magic happens when clicking on the image / image buttons.

abidlabs commented 1 year ago

I'm a little hesitant to expand the API of the gr.Chatbot to include so many LaTeX-specific parameters. What about if you use code brackets instead, like this:

import gradio as gr

with_code = """
The equation is: 

$$E = m \cdot c^2$$

"""

without_code = """
The equation is: 
$$E = m \cdot c^2$$
"""

with gr.Blocks() as demo:
    gr.Chatbot([
        [with_code,
        without_code]
    ])

demo.launch()

This produces:

image

You even get the nice copy button automatically next to the code block

GaiZhenbiao commented 1 year ago

What we really want is being able to switch between rendered text and raw text. Currently Gradio renders everything, but with ignoredClasses we can specify which part should be rendered and which part shouldn't. It's a genuinely useful parameter.

Keldos-Li commented 1 year ago

Sorry for not replying for so long. I was recently outside and contracted COVID-19. Now I am able to revisit this issue.

I understand that you believe gradio should not introduce unnecessary parameters... and for framework builders, that is indeed the case... but latex-related matters may be an exception.

We cannot have formulas appear in code blocks. Unlike debugging, our chatbot's responses are solely from AI, and we cannot predict what the AI will output or make it use code blocks to format the formulas in a standardized way. We also cannot post-process the formulas. (Obviously, the same is true for the use cases of most other users, so it is unrealistic to specify how to write the formula.) Therefore, the solution you suggested is not feasible.

In fact, our helplessness started when you switched to using frontend rendering for the chatbot's content. What we have been wanting from the beginning is a method that allows us to preserve the original output text in the chatbot without any rendering, whether it be markdown or latex rendering...

So it would also be nice if you add a gradio parameter called ignoredClassess or something else where we can set custom CSS class, not for katex only, but for the whole chatbot, and in these classes not a single character will be rendered.

abidlabs commented 1 year ago

Sorry for the late response @Keldos-Li and I hope you have recovered fully. I really think that adding ignoredClasses is beyond the scope of the humble gr.Chatbot component. We are working towards allowing the community to create custom components and this could be a good use case for that. I'll go ahead and label this issue accordingly

Keldos-Li commented 1 year ago

Thanks, I totally understand why you think some parameters shouldn't be added to gr.Chatbot. Looking forward to your custom component features!

Keldos-Li commented 1 year ago

Good news, I think we have finally found a reasonable way to use JavaScript to simultaneously render KaTeX and not render LaTeX formulas in the message. I believe this is a feasible workaround, so maybe this issue could be closed...

iShot_2023-08-16_上午12 48 49

(I noticed you added copy button in 3.40.0, emm, we will continue to use our old button though)

By the way, the new chatInterface looks great, but it seems that the chatbot had more flexibility. I think we will continue using the chatbot component for our development. Thank you, and thanks to Gradio!

abidlabs commented 1 year ago

Nice glad to hear it! Yes ChatInterface is designed to get a chat user interface up and running very quickly, but for more custom uses, we recommend using the Chatbot directly. Nice demo!

I'll go ahead and close this