Bad handling of \n in chatbot

[pseudotensor]

Describe the bug

Real text output looks like:

 It's hard to say who is smarter between Isaac Newton and Albert Einstein, as both were incredibly intelligent individuals with unique perspectives on the world around them. However, here are some of the key differences between the two that may help you make a decision:

Newton was

but in gradio chatbot it looks like:

It's hard to say who is smarter between Isaac Newton and Albert Einstein, as both were incredibly intelligent individuals with unique perspectives on the world around them. However, here are some of the key differences between the two that may help you make a decision:
Newton

i.e.:

Even though when I copy-paste the text from the gradio UI, it somehow expands as text to be correct:

It's hard to say who is smarter between Isaac Newton and Albert Einstein, as both were incredibly intelligent individuals with unique perspectives on the world around them. However, here are some of the key differences between the two that may help you make a decision:

Newton

I've noticed alot of variability in the handling of \n. In prior gradio==3.31.0, I could convert "\n" to "
" and it fixed the problem.

However, in gradio==3.32.0 this no longer works, and the "
" show up literally in the chat output. I have not found any work around.

I debugged the code itself, and the "processed_messages" still looks fine. So it's "just" the UI itself that is wrongly displaying \n. I presume this is related to how new lines in html don't directly matter, but
showing up literally in chatbot now is new to me.

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

As above

Screenshot

Logs

NA

System Info

gradio==3.32.0

Severity

blocking upgrade to latest gradio version

[pseudotensor]

Related, a breaking condition with new gradio, is that all html no longer renders:

https://github.com/gradio-app/gradio/issues/4345

[dawoodkhan82]

@pseudotensor to clarify the issue. Do you expect \n to do nothing? because right now it creates a new line

[pseudotensor]

As you can see in my first example, \n\n is becoming \n. The UI is not handling. I expect \n\n to look like 2 new lines, not just 1. A actual copy-pasted text from the UI is correct, however.

In newer gradio, things were broken, but that's being solved separately.

[abidlabs]

I can confirm the issue @dawoodkhan82. Here's a simpler repro:

import gradio as gr

message = [(
    "Single newline:\nMessage continues here",
    "Double newline:\n\nMessage continues here",    
)]

with gr.Blocks() as demo:
    gr.Chatbot(message)

demo.launch()

As @pseudotensor mentioned, interestingly when you copy-paste the text, the double new line appears again. I think this might be our markdown processor in the frontend collapsing double new lines

[dawoodkhan82]

@abidlabs @pseudotensor afaik multiple empty lines are not supported in markdown using \n. You can use <br/> to create multiple new lines. We used to manually convert \n to <br/> in the backend for chatbot. I don't know if we should bring that back, since it goes against markdown syntax.

[abidlabs]

Ah that explains it. I think it's fine to leave as is in that case. @pseudotensor you can convert "\n" to "
" as part of your function that generates the message for the Chatbot.

[abidlabs]

Can confirm that works as expected here:

import gradio as gr

message = [(
    "Single newline:\nMessage continues here".replace("\n", "<br/>"),
    "Double newline:\n\nMessage continues here".replace("\n", "<br/>"),    
)]

with gr.Blocks() as demo:
    gr.Chatbot(message)

demo.launch()

[pseudotensor]

Yes, if you see my first comment, I was doing that, but the new gradio==3.32.0 broke that capability, but that had all sorts of other issues with rendering, so hopefully that is fixed.

Are all the rendering regressions fixed?

[abidlabs]

Yes as far as we're aware they are fixed, so you should be able to just convert \n to <br> in your function and get the desired behavior. Let us know if you experience any trouble. @dawoodkhan82 I think we can close this issue

[pseudotensor]

Hi @abidlabs , this suggestion does not work in general. It fails in a code block, because then the <br> appears literally.

i.e. you will get:

I really think gradio should handle "properly" with normal expectations of what a string output would look like in these various cases. That is the purpose behind the fairly non-trivial markdown, tex, and other processing code in gradio is to handle these cases properly.

[abidlabs]

Ah so I looked into this a bit more, and it turns out that there is somewhat of a bug on the Gradio side. The HTML that is rendered by marked in this example:

import gradio as gr

message = [(
    "Single newline:\nMessage continues here",
    "Double newline:\n\nMessage continues here",    
)]

with gr.Blocks() as demo:
    gr.Chatbot(message)

demo.launch()

is actually different between the two example:

The first one just introduces a <br> while the second one actually has separate <p> for each of the two lines. However, we don't have any vertical margin between subsequent <p> tags so it appears as they are part of the same paragraph. @dawoodkhan82 let's go ahead and add some margin on top of every paragraph tag (excluding the first paragraph tag). I think that should do the trick.

[pseudotensor]

Thanks for looking into it!

[jerpint]

I think the newline bug is back. Using @abidlabs simple example:

import gradio as gr

message = [(
    "Single newline:\nMessage continues here",
    "Double newline:\n\nMessage continues here",
)]

with gr.Blocks() as demo:
    gr.Chatbot(message)

demo.launch()

Gives the following (the first \n is ignored):

MacOS, env:

gradio==3.45.1
gradio_client==0.5.2

[pseudotensor]

3.41.0:

3.45.1:

[abidlabs]

Aggh this regression should not have happened, sorry folks. @dawoodkhan82 can you look into this & add a test to prevent regressions?

[pseudotensor]

Seems to have started in 3.42.0 . 3.41.2 is fine. I didn't notice until I was planning on upgrading and hit too.

[pseudotensor]

@abidlabs Any possible css work-around?

[pseudotensor]

In case useful to know, using the copy-paste button in the UI leads to same exact result in both cases. But rendering is different.

For old gradio that looks correct, a direct copy-paste of the rendered part yields:

On my machine, the results are:

Merge lists: 0.256 seconds
Extend list: 0.184 seconds
Plus equal: 0.208 seconds
Sort then concat: 1.092 seconds

So, the extend method is the fastest, followed by the += operator, and the merge function is the slowest. The sort method is much slower than the other methods because it has to sort the entire list twice.

while on new gradio such a copy paste yields:

On my machine, the results are:

Merge lists: 0.256 seconds Extend list: 0.308 seconds Plus equal: 0.324 seconds Sort then concat: 1.372 seconds

So, the merge function is the fastest method for merging two sorted lists, followed closely by the extend method. The plus_eq method is slightly slower, and the sort_then_concat method is the slowest.

[abidlabs]

So the issue here is that this is actually the correct behavior according to the Markdown Common Spec: https://spec.commonmark.org/0.30/#soft-line-breaks -- single new lines should be ignored.

One option that you have is to disable Markdown rendering via the render_markdown parameter, in which case the strings are just rendered as is.

To take @jerpint's example:

import gradio as gr

message = [(
    "Single newline:\nMessage continues here",
    "Double newline:\n\nMessage continues here",
)]

with gr.Blocks() as demo:
    gr.Chatbot(message, render_markdown=False)

demo.launch()

which produces:

Would that work for you @pseudotensor @jerpint?

[jerpint]

I would definitely prefer to preserve markdown rendering overall. I feel that currently the double newline feels like 2 line breaks instead of one, and that there is no way to get a "single line break" type of look.

Also chatGPT tends to add single /n as meaning "new paragraph", which gets lost in this current setup , so not sure what the best solution is here. The behaviour before this "regression" felt more natural

[dawoodkhan82]

and that there is no way to get a "single line break" type of look.

So actually if you add <br> in place of \n it does create a new line.

[jerpint]

I guess my expectation would be to render as close to what you expect a typical LLM to render - if chatGPT's UI renders /n as a newline it would probably be better for this one to do the same? It might lead to less "optimal" outputs otherwise ?

[pseudotensor]

@abidlabs Generally disabling markdown rendering won't work. IMO, it's gradio's job to consume (nominally) input text and render it as expected as best as expected, not follow a strict spec. Gradio's history has been to render as markdown if can in backend if the thing itself is markdown. This rendering was pushed to front end. But goal was never to strictly adhere to markdown spec and expect user to input strictly only markdown.

@dawoodkhan82 As mentioned in above conversations, replacing does not work. It breaks code blocks etc. Hacking that may be possible, but it'll likely always be incomplete.

I'd expect gradio to try to render as faithfully as possible. i.e. text is text, and markdown (like code blocks) are rendered as expected, images as images, etc. I don't think anyone expects to have to push to the chatbot strict markdown.

Also, I don't fully follow the variances here. For a while it was working fine. Only in 3.42.0+ did it get broken. So was markdown not strictly adhered to before? I don't fully follow.

[jerpint]

Very early versions of gradio used to require \<br> for single line breaks and it led to all sorts of awkwardness.

For example; if I just put a simple rule, e.g. "replace \n with \<br>"

If chatGPT produced

'''\nvalid code'''

Well now I have to add a rule to my replacement to ignore /n to \<br> inside code blocks which can be error prone

I think that for most use cases users will expect /n to render as a single new line, and explains the nature of this very issue, which was once open and closed once the issue was resolved.

If it's really important to also adhere to the markdown spec, perhaps a flag to enable that could be a solution, being disabled by default?

[abidlabs]

Hi folks, we discussed internally and agreed with your suggestion of reverting the change.

By default, the gr.Chatbot will render single new lines as new lines (so render Markdown the way that GitHub does it as opposed to Common Spec). Same for gr.DataFrame. On the other hand, gr.Markdown will continue rendering Markdown according to Common Spec, the way it has always rendered Markdown. We've also introduced a line_breaks parameter to all three components for developers to be able to control this behavior as desired.

See: https://github.com/gradio-app/gradio/pull/5755, which should be out soon.

[pseudotensor]

@abidlabs I'm seeing the same issue again on 3.49.0.

E.g. text looks like this in UI:

But text actually outputted is:

Robust Speech Recognition via Large-Scale Weak Supervision
===============================================

Introduction
------------

Speech recognition is a field of artificial intelligence (AI) that deals with the conversion of spoken language into text. Speech recognition systems can be used in a variety of applications, including virtual assistants, voice-activated devices, and automated customer service.

One of the challenges in speech recognition is dealing with noisy audio data. Noise can come from a variety of sources, including background noise, reverberation, and speaker variations. One way to address this challenge is through the use of weakly supervised learning techniques.

Weakly supervised learning involves training a model with limited labeled data and a large amount of unlabeled data. In the case of speech recognition, this can involve using a combination of labeled and unlabeled data to improve the performance of the model.

Data Collection
----------------

To collect the data needed for weakly supervised speech recognition, we can use a combination of publicly available datasets and crowdsourcing platforms. Publicly available datasets can provide a large amount of labeled data, while crowdsourcing platforms can provide a large amount of unlabeled data.

Data Cleaning
------------

Once the data has been collected, it is important to clean it to remove any errors or inconsistencies. This can involve tasks such as data normalization, data augmentation, and data preprocessing.

Model Selection
--------------

There are many different models that can be used for speech recognition, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and transformers. The choice of model will depend on the specific requirements of the application.

Training the Model
-----------------

Once the data has been collected and cleaned, the next step is to train the model. This can involve techniques such as transfer learning, data augmentation, and regularization.

Evaluation Metrics
----------------

To evaluate the performance of the speech recognition model, we can use metrics such as word error rate (WER) and character error rate (CER). These metrics can provide insight into the accuracy of the model and help identify areas for improvement.

Conclusion
----------

In conclusion, weakly supervised speech recognition via large-scale weak supervision can be achieved through the use of labeled and unlabeled data, data cleaning, model selection, training the model, and evaluation metrics. With the right approach, it is possible to create highly accurate speech recognition systems that can be used in a variety of applications.

I can see the new line generated and then deleted as pure UI issue.

Do you need a repro? Should be obvious.

Perhaps I misunderstand how things are done in gradio and its interpreting the structure as markdown, as it's also missing all the -- == stuff.

That is, it is removing \n\n -> \n and removing all --- and ===

[abidlabs]

Hmm it looks like this is a separate issue coming from the use of ====== and -----.

Without that, I can't repro the issue. For example:

import gradio as gr

message = [(
    "Single newline:\nMessage continues here",
    "Double newline:\n\nMessage continues here",
)]

with gr.Blocks() as demo:
    gr.Chatbot(message)
    gr.Chatbot(message, line_breaks=False)
    gr.Chatbot(message, line_breaks=True)

demo.launch()

In all 3 cases, double new lines are rendered correctly. If that's what you're seeing as well, could you create a new issue please @pseudotensor?

[pseudotensor]

Ok, perhaps the --- and === are causing the \n\n -> \n Will open new issue