UI hangs for chatbot when streaming in some cases -- with a repro

pseudotensor commented 1 year ago

Describe the bug

Once html part starts, everything gets stuck. Even through a next response starts freshly yield back to the text box, it never recovers.

The html is perfectly valid once fully rendered, but in any case the 3rd block rendering should have started freshly.

In addition, it shouldn't hang the UI after being done with yielding and doing the return.

Note that gradio renders the html fine if I change the code to render the html in a single go.

However, in my case this doesn't work for whatever reason, and instead the hang happens just after the "sources" block is rendered, and the 3rd response never happens and the hang stays forever.

So I'm giving a repro you might fake a work-around for, but that won't work in general. There must be some more general issue that makes the UI hang.

Have you searched existing issues? 🔎

[X] I have searched and found no existing issues

Reproduction

Choose textbox of 0.001 . Once html hit, text goes way. Once html is done being pushed with extra spaces, it should be wiped clean by the 3rd response that starts over with new text, but instead the UI hangs.

import gradio as gr
import time

response1 = """The most popular Coca-Cola products are the classic Coca-Cola soda, Diet Coke, Coca-Cola Zero Sugar, and Sprite. Coca-Cola, with its iconic red label and refreshing taste, has been a fan favorite for decades. It is loved for its perfect balance of sweetness and carbonation, making it a go-to choice for many. Diet Coke, on the other hand, appeals to those who prefer a sugar-free option without compromising on the great taste. Coca-Cola Zero Sugar offers a similar experience to regular Coca-Cola but with zero calories. Lastly, Sprite, with its lemon-lime flavor and crisp bubbles, is another highly popular choice among consumers. Its refreshing and tangy taste makes it a preferred option for those seeking a citrusy twist. These four Coca-Cola products have consistently topped the charts in terms of popularity and continue to be enjoyed by millions worldwide."""

response2 = """Based on the information provided, it is not explicitly stated which Coca-Cola products are the most popular. The document primarily focuses on financial results, volume growth, and operational performance of Coca-Cola FEMSA, the largest Coca-Cola franchise bottler in the world. It does not provide specific details about the popularity of individual Coca-Cola products.

<details><summary><font size="2">Sources</font></summary><font size="2"><font size="2">Sources [Score | Link]:<ul></font><font size="2"><li>0.82 | <font size="2"><a href="file//tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf" target="_blank"  rel="noopener noreferrer">/tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf</a></font></li><details><summary><font size="2">and distributes trademark beverages of T</font></summary><font size="2">and distributes trademark beverages of The Coca-Cola Company, offering a wide portfolio of 131 brands to a population of more
than 266 million. With over 80 thousand employees, the Company markets and sells approximately 3.5 billion unit cases
through 2 million points of sale a year. Operating 49 manufacturing plants and 260 distribution centers, Coca-Cola FEMSA is
committed to generating economic, social, and environmental value for all of its stakeholders across the value chain. The</font></details></font><font size="2"><li>0.78 | <font size="2"><a href="file//tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf" target="_blank"  rel="noopener noreferrer">/tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf</a></font></li><details><summary><font size="2">Coca-Cola FEMSA Reports 1Q23 Results Apr</font></summary><font size="2">Coca-Cola FEMSA Reports 1Q23 Results
April 26, 2023
Page 13 of 14
(1)
Volume is expressed in unit cases. Unit case refers to 192 ounces of finished beverage product (24 eight-ounce servings) and, when applied to soda fountains, refers to
the volume of syrup, powders, and concentrate that is required to produce 192 ounces of finished beverage product.
(2)</font></details></font><font size="2"><li>0.77 | <font size="2"><a href="file//tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf" target="_blank"  rel="noopener noreferrer">/tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf</a></font></li><details><summary><font size="2">Coca-Cola FEMSA Reports 1Q23 Results Apr</font></summary><font size="2">Coca-Cola FEMSA Reports 1Q23 Results
April 26, 2023
Page 8 of 14
DEFINITIONS
Volume is expressed in unit cases. Unit case refers to 192 ounces of finished beverage product (24 eight-ounce servings) and,
when applied to soda fountains, refers to the volume of syrup, powders, and concentrate that is required to produce 192
ounces of finished beverage product.
Transactions refers to the number of single units (e.g., a can or a bottle) sold, regardless of their size or volume or whether they</font></details></font><font size="2"><li>0.77 | <font size="2"><a href="file//tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf" target="_blank"  rel="noopener noreferrer">/tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf</a></font></li><details><summary><font size="2">Coca-Cola FEMSA Reports 1Q23 Results Apr</font></summary><font size="2">Coca-Cola FEMSA Reports 1Q23 Results
April 26, 2023
Page 2 of 14
Mexico City, April 26, 2023, Coca-Cola FEMSA, S.A.B. de C.V. (BMV: KOFUBL, NYSE: KOF) (“Coca-Cola FEMSA”,
“KOF” or the “Company”), the largest Coca-Cola franchise bottler in the world by sales volume, announces results
for the first quarter of 2023.
•
Volume growth 6.6%
•
Revenue growth 12.0%
•
Operating income growth 12.9%
•
Majority net income growth 35.3%
•</font></details></font><font size="2"><li>0.77 | <font size="2"><a href="file//tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf" target="_blank"  rel="noopener noreferrer">/tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf</a></font></li><details><summary><font size="2">Coca-Cola FEMSA Reports 1Q23 Results Apr</font></summary><font size="2">Coca-Cola FEMSA Reports 1Q23 Results
April 26, 2023
Page 10 of 14
(1)
Except volume and average price per unit case figures.
(2)
Please refer to page 13 for revenue breakdown.
(3)
Includes equity method in Jugos del Valle and Leão Alimentos, among others.
(4)
Includes equity method in PIASA, IEQSA, Beta San Miguel, IMER, and KSP Participacoes, among others.
(5)
The operating income and EBITDA lines are presented as non-GAAP measures for the convenience of the reader.
(6)</font></details></font><font size="2"><li>0.76 | <font size="2"><a href="file//tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf" target="_blank"  rel="noopener noreferrer">/tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf</a></font></li><details><summary><font size="2">and regulations of the SEC (that apply t</font></summary><font size="2">and regulations of the SEC (that apply to foreign private issuers) and of the BMV. Filings we make electronically with the SEC and
the BMV are available to the public on the Internet at the SEC’s website at www.sec.gov, the BMV’s website at
www.bmv.com.mx, and our website at www.coca-colafemsa.com.
Coca-Cola FEMSA, S.A.B. de C.V. is the largest Coca-Cola franchise bottler in the world by sales volume. The Company produces</font></details></font><font size="2"><li>0.76 | <font size="2"><a href="file//tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf" target="_blank"  rel="noopener noreferrer">/tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf</a></font></li><details><summary><font size="2">Transactions refers to the number of sin</font></summary><font size="2">Transactions refers to the number of single units (e.g., a can or a bottle) sold, regardless of their size or volume or whether they are sold individually or in multipacks,
except for soda fountains, which represent multiple transactions based on a standard 12 oz. serving.
YoY
Sparkling
Water (1)
Bulk (2)
Stills
Total
Sparkling
Water (1)
Bulk (2)
Stills
Total
Δ %
Mexico (3)
309.5
26.4
87.6
35.2
458.8
301.9
21.3
67.5
32.7
423.5
8.3%
Guatemala
35.6
1.4
-
2.3
39.2
30.6
1.1
-
1.9
33.7
16.3%
CAM South
31.4
2.0</font></details></font><font size="2"><li>0.76 | <font size="2"><a href="file//tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf" target="_blank"  rel="noopener noreferrer">/tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf</a></font></li><details><summary><font size="2">Coca-Cola FEMSA Reports 1Q23 Results Apr</font></summary><font size="2">Coca-Cola FEMSA Reports 1Q23 Results
April 26, 2023
Page 3 of 14
RECENT DEVELOPMENTS
•
On March 27, 2023, Coca-Cola FEMSA held its Annual Ordinary General Shareholders’ Meeting, during which its
shareholders approved among other things, the Company’s consolidated financial statements for the year ended
December 31, 2022, the annual report presented by the Board of Directors, the declaration and payment of</font></details></font><font size="2"><li>0.76 | <font size="2"><a href="file//tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf" target="_blank"  rel="noopener noreferrer">/tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf</a></font></li><details><summary><font size="2">Coca-Cola FEMSA Reports 1Q23 Results Apr</font></summary><font size="2">Coca-Cola FEMSA Reports 1Q23 Results
April 26, 2023
Page 14 of 14
LTM
1Q23
Mexico
7.12%
0.96%
Colombia
13.64%
4.22%
Brasil
5.56%
1.63%
Argentina
108.32%
21.28%
Costa Rica
5.70%
-0.35%
Panama
1.91%
1.11%
Guatemala
10.26%
1.95%
Nicaragua
10.92%
1.50%
Uruguay
7.58%
3.09%
(1) Source: inflation estimated by the company based on historic publications from the Central Bank of each country.
1Q23
1Q22
Δ %
México
18.70
20.52
-8.9%
Colombia
4758.63
3914.87
21.6%
Brasil
5.19
5.23
-0.7%
Argentina
192.41
106.58
80.5%</font></details></font><font size="2"><li>0.76 | <font size="2"><a href="file//tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf" target="_blank"  rel="noopener noreferrer">/tmp/gradio/16b4ecf02adc0d2832ca016668ab7646371009ef/Coca-Cola-FEMSA-Results-1Q23-vf-2 1.pdf</a></font></li><details><summary><font size="2">Coca-Cola FEMSA Reports 1Q23 Results Apr</font></summary><font size="2">Coca-Cola FEMSA Reports 1Q23 Results
April 26, 2023
Page 6 of 14
MEXICO & CENTRAL AMERICA DIVISION FIRST QUARTER RESULTS
(Mexico, Guatemala, Costa Rica, Panama, and Nicaragua)
Volume increased 8.8% driven by growth across all territories in the division. These volumes include the integration of Cristal
bulk water business. Excluding these volumes, volume would have increased 5.7% in the division and 4.8% in Mexico.</font></details></font>Total document chunks used: 30<p><font size="2"></ul></p>End Sources<p></font></font></details>
"""

response3 = """Based on the information provided, it is not explicitly stated which Coca-Cola products are the most popular. The document primarily focuses on financial results, volume growth, and operational performance of Coca-Cola FEMSA, the largest Coca-Cola franchise bottler in the world. It does not provide specific details about the popularity of individual Coca-Cola products."""

def bot(msg, history):
    history = [[None, None]]
    history[-1][0] = "Which coca cola products are most popular?"

    for response in [response1, response2, response3]:
        response_words = response.split(' ')
        history[-1][1] = ""
        t0 = time.time()
        for word in response_words:
            history[-1][1] += word + ' '
            time.sleep(float(msg))
            yield msg, history
            print('%s %s' % (time.time() - t0, history), flush=True)

with gr.Blocks() as demo:
    chatbot = gr.Chatbot(
        [],
    )
    msg = gr.Textbox()
    clear = gr.ClearButton([msg, chatbot])
    msg.submit(bot, [msg, chatbot], [msg, chatbot])

demo.queue()
if __name__ == "__main__":
    demo.launch()

Screenshot

valid html:

Logs

No response

System Info

gradio==3.49.0
gradio_client==0.6.1

Severity

Blocking usage of gradio

pseudotensor commented 1 year ago

More generally, I've seen spurious hangs in UI for no apparent reason, that I think must be related to what happens in this repro.

pseudotensor commented 11 months ago

I think hangs some are happening because the UI is lagging, a related issue. Basically I can see that the UI lags behind the back end for streaming. The BE gets done long before the UI. However, once the BE is done and stop iteration is hit, the UI also stops.

It's like the signal to stop overrides any remaining UI renderings, and further makes the UI hang as it is some kind of mixed state according to the UI gradio code.

I can reproduce this particular scenario over and over.

pseudotensor commented 11 months ago

Any update? I gave full repro.

pseudotensor commented 10 months ago

Any update? I gave full repro.

freddyaboulton commented 10 months ago

I think the issue is that during the iteration of response_2 the html string is malformed or something like that and that's what's causing the frontend to crash/hang.

gradio-app / gradio