Document the image formats that are acceptable in the `Image` and `Gallery` components

[vadimkantorov]

Describe the bug

I'm interested in having animated small videos at display in Image and in Gallery (in gallery it's also useful to have an option to synchronize all GIFs and pad all GIFs to maximum number of frames).

We can often have a GIF as result of matplotlib FuncAnimation, it's simpler to manage than video. So it would be nice to have these GIFs displayed as video too. I can either provide these GIFs as paths or read the video frames somehow into an array. It can be nice to have the browser display render them as GIFs specifically. If there's many of them, it should create less load for the browser.

I found https://github.com/gradio-app/gradio/issues/577 but could not understand if displaying these GIFs is supported as Videos and whether I can provide a THW3 arrays anywhere to Image/Gallery items.

Thanks!

Have you searched existing issues? 🔎

• [X] I have searched and found no existing issues

Reproduction

N/A

Screenshot

No response

Logs

No response

System Info

N/A

Severity

I can work around it

[abidlabs]

Hi @vadimkantorov you can already display gifs using the gr.Image() and gr.Gallery() components. You can verify with a simple demo like this:

import gradio as gr

with gr.Blocks() as demo:
    gr.Gallery([
        "example.gif",
        "example.gif"
    ])
    gr.Image("example.gif")

demo.launch()

[abidlabs]

Am I misunderstanding something?

[vadimkantorov]

Will these be rendered as GIFs in the browser? Or converted to videos?

I guess the remaining bit is being able to pass THWC and have it rendered either as GIF or as a video (depending on passed option).

Can I pass some individual text image labels to the gallery? Currently even image file paths aren't shown as tooltips.

[abidlabs]

They will be rendered as gifs. If you'd like to convert to videos, you can use a library like moviepy to do so: https://github.com/gradio-app/gradio/issues/5174

Not sure what THWC is.

Yes, you can pass in image captions to the gallery, using a tuple structure: (img_path, img_caption).

[vadimkantorov]

THWC meaning numpy array shape Time x Height x Width x Channels (Channels is usually 3)

One unrelated feedback - it would be great if button Enter by default was hooked to the orange default button "Submit"

Also, would be good to have an option to open an image / image from Gallery to the full screen

[vadimkantorov]

Maybe docs (https://www.gradio.app/docs/image, https://www.gradio.app/docs/gallery) should explicitly mention the formats that they support. E.g. is SVG supported? Is PDF supported (sometimes used for plots)?

[abidlabs]

A lot of different issues here, but I think the most pertinent is to mention the image formats that are acceptable in the Image and Gallery components -- could you take a look @aliabd?

I'll rename the issue to reflect this focus.

One unrelated feedback - it would be great if button Enter by default was hooked to the orange default button "Submit"

For gr.Interface this is indeed the case already. If you are using gr.Blocks(), you can achieve this by using the .submit() event listener for the button.

[vadimkantorov]

I am using gradio.Interface, and hitting Enter does nothing (in Chrome) :(

[abidlabs]

Are you using the latest version of gradio? What demo are you trying?

This works for me:

import gradio as gr

gr.Interface(lambda x:x, "textbox", "textbox").launch()

[vadimkantorov]

>>> gradio.__version__
'3.39.0'

I'm using my custom code https://github.com/vadimkantorov/selective_search_pytorch/blob/master/demo.py#L236 :

import gradio
gradio_inputs = [
    gradio.Image(label = 'input image', value = args.input_path),
    gradio.Radio(label = 'preset', choices = ['fast', 'quality', 'single'], value = 'single'),
    gradio.Radio(label = 'algo', choices = ['pytorch', 'opencv', 'opencv_custom'], value = 'pytorch'),
    gradio.CheckboxGroup(label = 'visualizations', choices = ['instance grids', 'merging segments', 'merging trees']),
    gradio.Checkbox(label = 'remove duplicate boxes'),
    gradio.Checkbox(label = 'profile'),
    gradio.Number(label = 'beginboxind', minimum = 0, precision = 0, value = 0),
    gradio.Number(label = 'endboxind', minimum = 0, precision = 0, value = 64),
    gradio.Number(label = 'grid', minimum = 0, precision = 0, value = 4),
    gradio.Number(label = 'seed', minimum = 0, precision = 0, value = 42),
]
gradio_outputs = [
    gradio.Textbox(label = 'log'),
    gradio.Number(label = 'num boxes', minimum = 0, precision = 0, value = 0),
    gradio.Image(label = 'top boxes'),
    gradio.Gallery(label = 'instance grids'),
    gradio.Gallery(label = 'merging segments'),
    gradio.Gallery(label = 'merging trees')
]
gradio_submit = lambda img_rgbhw3_255, preset, algo, visualizations, remove_duplicate_boxes, profile, beginboxind, endboxind, grid, seed: main(...)
gradio_demo = gradio.Interface(gradio_submit, inputs = gradio_inputs, outputs = gradio_outputs, flagging_dir = args.output_dir, allow_flagging = 'never')
gradio_demo.launch()

[vadimkantorov]

This works for me:

This works only if you first focus on the textbox manually (and the browser doesn't focus on it automatically).

E.g. I expect that running import gradio as gr; gr.Interface(lambda x:"foo", "textbox", "textbox").launch(), navigating to http://127.0.0.1:7860, I can immediately hit Enter and see foo. This does not happen (only happens if I focus om the textbox first with moving the mouse and clicking). E.g. I would like to get some default action triggered without moving the mouse/clicking/focusing on an element.

[abidlabs]

Ah yeah we don't enable autofocus by default for accessibility reasons. But if you really want to to do that, pass in the autofocus=True argument to your Textbox. E.g.

import gradio as gr

gr.Interface(lambda x:x, gr.Textbox(autofocus=True), "textbox").launch()

[vadimkantorov]

Yeah, this could be a workaround, but best would be that Enter gets handled even if no element on the page is autofocus'd

E.g. I have an Image element (with some default image pre-selected). Image does not support autofocus kwarg

In fact plenty of elements don't support autofocus kwarg

[vadimkantorov]

In fact, in my example I have no input elements that support autofocus. And using autofocus on an output textbox element does nothing

[abidlabs]

Ah yeah I see -- that's something we can consider. Feel free to open a separate issue for this.