Demo crushes given many images + text in unwrapped only when deployed

[yonatanbitton]

Describe the bug

Hi. I have the following app in huggingface: link.

It's a 25 rows * 4 columns of images. I have 500 images, but when I try to increase the rows I receive the following error, which occurs when deploying the app, and also locally, tested with two different laptops:

Traceback (most recent call last):
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1374, in getresponse
    response.begin()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/socket.py", line 705, in readinto
    return self._sock.recv_into(b)
TimeoutError: timed out

The second problem is unwrapped text when deployed. When running locally, the text block shows all of the text. However, when deployed, the text is cut an "unwrapped" and it's not possible to see all of it without scrolls.

How can I fix these issues? Thanks!

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

https://huggingface.co/spaces/nlphuji/whoops_explorer

Screenshot

Logs

/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/bin/python /Users/yonatanbitton/PycharmProjects/wmtis-explorer/app.py 
Found cached dataset whoops (/Users/yonatanbitton/.cache/huggingface/datasets/nlphuji___whoops/TEST/1.1.0/f28e9b5029c9acfe5ad0a0c0c579164280e75af77aa23149ee1c46f2b6a66f0d)
100%|██████████| 1/1 [00:00<00:00, 287.77it/s]
Loaded WMTIS, first example:
{'image': <PIL.PngImagePlugin.PngImageFile image mode=RGBA size=1024x1024 at 0x158CF6D70>, 'designer_explanation': 'Chili peppers are very spicy, and toddlers usually do not eat spicy food.', 'selected_caption': 'A baby eating a hot chili pepper', 'crowd_captions': ['A child chews on a red pepper.', 'A young child snack on a red chili pepper.', 'A baby eating a red pepper.', 'A baby eating a hot chili pepper', 'A young child eats a red pepper'], 'crowd_explanations': ["Red peppers are spicy and could burn a child's mouth, instead, a toddler would want a sweet confectionary.", "Chili peppers are generally hot, or spicy, and most small children don't like spicy foods.", "Babies don't usually have a wide palette and wouldn't like eating a pepper.", "Babies have uninitiated palates and are used to bland foods, so they wouldn't be able to tolerate the heat of a hot pepper like an experienced adult would.", "Red peppers are spicy and could cause discomfort or burning sensations in a child's mouth, therefore a sweet treat would be more appealing to a toddler"], 'crowd_underspecified_captions': ['A child is given a treat.', 'A young child has a vegetable for a snack.', 'A person eating a red pepper.', 'A baby eating finger food', 'A child eating a red snack'], 'question_answering_pairs': [['What is shown eating a red pepper?', 'A baby'], ['Who eats a red pepper?', 'A young child'], ['What is the baby eating in the picture?', 'a hot chili pepper'], ['What does a young child snack on?', 'a red chili pepper'], ['What does the child chew on?', 'a red pepper']], 'commonsense_category': 'Age mismatch', 'image_id': '1313395dfd6f998b1029f676ca336966c1583eb727f99af05a010c028632527f', 'image_designer': 'Ady Bertschneider'}
all dataset size: 500
Traceback (most recent call last):
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1374, in getresponse
    response.begin()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/socket.py", line 705, in readinto
    return self._sock.recv_into(b)
TimeoutError: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/lib/python3.10/site-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 451, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 340, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='127.0.0.1', port=7860): Read timed out. (read timeout=3)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/app.py", line 82, in <module>
    demo.launch()
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1459, in launch
    if _frontend and (not networking.url_ok(self.local_url)) and (not self.share):
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/lib/python3.10/site-packages/gradio/networking.py", line 181, in url_ok
    r = requests.head(url, timeout=3, verify=False)
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/lib/python3.10/site-packages/requests/api.py", line 100, in head
    return request("head", url, **kwargs)
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/Users/yonatanbitton/PycharmProjects/wmtis-explorer/venv/lib/python3.10/site-packages/requests/adapters.py", line 578, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='127.0.0.1', port=7860): Read timed out. (read timeout=3)

Process finished with exit code 1

System Info

Locally, MacBook Pro, gradio version '3.20.0'

Severity

annoying

[yonatanbitton]

Hi, is there an update regarding this issue? Why does it crush above 100 images? Is there a better "gradio element" to use in order to show this gallery? Thanks

[abidlabs]

Hi @yonatanbitton I can't open the Space for testing (just loads forever for me). However, have you tried the gr.Gallery() component to display the 100 images?

[yonatanbitton]

Hi, the space works, you should just wait a bit. That might also be a cause of the problem. We tried gr.Gallery, but we also have annotations for each image. Is it possible to include some annotations? Example of the annotation for a given image below (and another problem is that it is cropped).

[abidlabs]

You can include captions with the Gallery component: just return each file along with its caption in a tuple of two strings like this: (filepath, caption) and the caption will show up at the bottom of the image.

[yonatanbitton]

Is there an option for more than one captions? We have 15 captions and 10 VQA instances we'd like to show, along with some additional metadata. The current grid we have is perfect, but the problem is that it crushes without a logical error message. Could we maybe understand why does it crush? Thanks 🙏

[abidlabs]

Oh no sorry the Gallery only supports 1 caption per image. Ok so I'm able to run your Space (after waiting a few seconds).

Just to confirm, the problem happens when you replace:

rows_number = 25

with

rows_number = int(dataset_size / columns_number)

is that right?

[abidlabs]

For the second issue, can you try setting the max_lines parameter in Textbox? Depending on the value set to max_lines, you'll get different behaviors:

import gradio as gr

with gr.Blocks() as demo:
    with gr.Accordion("test"):
        gr.Textbox("abc\nabc\nabc\nabc"*10, max_lines=1)
        gr.Textbox("abc\nabc\nabc\nabc"*10, max_lines=2)
        gr.Textbox("abc\nabc\nabc\nabc", max_lines=20)

demo.launch()

produces this:

[yonatanbitton]

@abidlabs for the second issue, i've tried your suggestion bad sadly it doesn't work. You can see the try and the corresponding result here.

For the first problem, yes, the large number of images is probably the cause, and what you wrote does reproduce the error. We have tried this great suggestion to add pagination, ideally it would be great to present 20 images in each time, and in this way to iterate the dataset. We created this dataset viewer to try and handle the problem: link. It shows how to iterate 20 samples in each time. However, we want to combine our original explorer (with the problem of loading too many images) with this last option of pagination. We started to try and do it, but we are not sure how to continue: Link. The question is what should be the inputs and outputs to the b1 and b2 buttons in order to show 20 samples in each time instead of the full dataset. They showed using an HTML layouts for it and we are afraid it might be too much for our use-case.

Pagination should be the answer, but the question is how to use it inside gradio. Thanks.

[abidlabs]

Hi @yonatanbitton for the max lines issue, it looks like you are using an older version of Gradio on the Space. If you go to the README and set the sdk_version to the latest version of Gradio (3.21.0), it works. I duplicated your Space to test, and you can see it working here: https://huggingface.co/spaces/abidlabs/whoops-explorer

For adding more rows dynamically, you can't do this right now with Gradio (though we are exploring it in #2566) -- for now, if you want to have the effect of pagination, you could replace all of the images / annotations when you click on the button. But you could also just get the same effect by refreshing the page, since your images are loaded randomly from the dataset so I don't know how useful such a button would be. As a small tip, I suggest maybe reducing the number of rows that are loaded initially, this should improve the loading time significantly and allow users to get a quick idea of what the dataset includes.

Hope that helps a bit, and congrats again on the launch of a very nice dataset.

[yonatanbitton]

Thanks! The lines number is fixed.

For the pagination, how can I make a button that will show the next 20 images? For example like this one, and this suggested solution? A "Next 20 image" button will do the work, since I want users to be able to iterate over all of the dataset (500 images) in this explorer... Thank you

[abidlabs]

Hi @yonatanbitton I put together a simple example using a mock dataset since that was a little easier to work with than the existing script. But the same principles apply to your app:

import gradio as gr

img1 = "https://images.unsplash.com/photo-1678854321097-6919ddad32ac?ixlib=rb-4.0.3&ixid=MnwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8&auto=format&fit=crop&w=693&q=80"
img2 = "https://images.unsplash.com/photo-1678569281064-87a3d8876cfc?ixlib=rb-4.0.3&ixid=MnwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8&auto=format&fit=crop&w=2127&q=80"

mock_dataset = []

for i in range(4):
    mock_dataset.append([img1, f"Image {i}"])
for i in range(4):
    mock_dataset.append([img2, f"Image {i+4}"])
for i in range(4):
    mock_dataset.append([img1, f"Image {i+8}"])
for i in range(4):
    mock_dataset.append([img2, f"Image {i+12}"])

with gr.Blocks() as demo:
    def render_sample(sample):
        image, text = sample
        with gr.Column():
            i = gr.Image(image)
            with gr.Accordion("Details"):
                t = gr.Textbox(text)
        return [i, t]

    def update_samples(i):
        return_values = []
        for j in range(4*i, 4*i+4):
            return_values.extend(mock_dataset[j])
        return_values.append(i+1)
        return return_values

    index = gr.State(1)
    components_list = []
    with gr.Row():
        for i in range(4):
            components = render_sample(mock_dataset[i])
            components_list.extend(components)

    btn = gr.Button("Next")
    btn.click(update_samples, index, [*components_list, index])

demo.launch()

Here's how it looks:

I'll go ahead and close this issue since for general questions like this (that are not feature requests or bug reports), a better place for them is GitHub Discussions or our Discord. (I'll close this issue). But hopefully this should give you a good idea of how to proceed.

cc @yvrjsharma @freddyaboulton as it relates to dataset exploration.

[ucas010]

@abidlabs hi,dear how to get 4 or more images from only 1 image? the 4 images are processed with different method from the one image . could you pls help me ? thx

[nivimahato]

Hi @abidlabs the above code runs perfectly but can we modify it to make user input the images instead I am trying below code for the same but gradio just hangs without display any error. Can you please help me identify why there is no error in gradio but also it doesnt through any error? import gradio as gr

Function to generate mock dataset based on image URLs

def generate_mock_dataset(img1, img2): mock_dataset = []

for i in range(4):
    mock_dataset.append([img1, f"Image {i}"])
for i in range(4):
    mock_dataset.append([img2, f"Image {i+4}"])

return mock_dataset

Function to render a sample

def render_sample(sample): image, text = sample print(image) with gr.Column(): i = gr.Image(image) print(i) with gr.Accordion("Details"): t = gr.Textbox(text) return [i, t]

Function to update samples

def update_samples(i): return_values = [] for j in range(4 i, 4 i + 4): return_values.extend(mock_dataset[j]) return_values.append(i + 1) return return_values

Gradio interface

with gr.Blocks() as demo: img1 = gr.Textbox(label="Image URL 1") img2 = gr.Textbox(label="Image URL 2") btn = gr.Button("Generate Mock Dataset")

index = gr.State(1)
components_list = []
def on_button_click(img1,img2):
    global mock_dataset
    mock_dataset = generate_mock_dataset(img1, img2)
    print(mock_dataset)
    with gr.Row():
        for i in range(4):
            print(i)
            components = render_sample(mock_dataset[i])
            print(components)
            components_list.extend(components)
    return components_list
btn.click(on_button_click,[img1,img2],[*component_list])

demo.launch(debug=True)