`gr.Image` with `tool=sketch` can receive wrong typed argument in preprocess

[space-nuko]

Describe the bug

Sometimes I can no longer send an image over to another image component, it causes an error

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

1. Upload image to first gr.Image on page
2. Send > Send2 > Send2 > Send

#!/usr/bin/env python

import gradio as gr
from PIL import Image

def copy_image(img):
    if isinstance(img, dict) and 'image' in img:
        return img['image']

    return img

with gr.Blocks() as demo:
    with gr.Column():
        input_image = gr.Image(label="Image for img2img", elem_id="img2img_image", show_label=False, source="upload", interactive=True, type="pil", tool="sketch", image_mode="RGBA").style(height=480)
        output_image = gr.Image(label="Image for inpainting with mask", show_label=False, elem_id="img2maskimg", source="upload", interactive=True, type="pil", tool="sketch", image_mode="RGBA").style(height=480)

    sendtobtn = gr.Button("Send")
    sendtobtn2 = gr.Button("Send2")

    sendtobtn.click(copy_image, inputs=[input_image], outputs=[output_image])
    sendtobtn2.click(copy_image, inputs=[output_image], outputs=[input_image])

demo.queue().launch(server_port=9876)

Screenshot

No response

Logs

Traceback (most recent call last):
  File "C:\Users\a\build\gradio-test\venv\lib\site-packages\gradio\routes.py", line 394, in run_predict
    output = await app.get_blocks().process_api(
  File "C:\Users\a\build\gradio-test\venv\lib\site-packages\gradio\blocks.py", line 1073, in process_api
    inputs = self.preprocess_data(fn_index, inputs, state)
  File "C:\Users\a\build\gradio-test\venv\lib\site-packages\gradio\blocks.py", line 962, in preprocess_data
    processed_input.append(block.preprocess(inputs[i]))
  File "C:\Users\a\build\gradio-test\venv\lib\site-packages\gradio\components.py", line 1598, in preprocess
    assert isinstance(x, dict)
AssertionError

(x is a 'str' in this case)

System Info

3.23.0, Windows, Chrome

Severity

blocking upgrade to latest gradio version

[space-nuko]

Also happens with gr.Interface examples

#!/usr/bin/env python

import os
import json

import numpy as np

import gradio as gr

def fn(im1, im2, im3, im4):
    return []

demo = gr.Interface(
    fn,
    inputs=[
        gr.Image(label="Image"),
        gr.Image(label="Image w/ Cropper", tool="select"),
        gr.Image(label="Image w/ Sketch", tool="sketch"),
        gr.Image(label="Image w/ Color Sketch", tool="color-sketch"),
    ],
    outputs=[
    ],
    examples=[
        [
            os.path.join(os.path.dirname(__file__), "images/cheetah1.jpg"),
            os.path.join(os.path.dirname(__file__), "images/cheetah1.jpg"),
            os.path.join(os.path.dirname(__file__), "images/cheetah1.jpg"),
            os.path.join(os.path.dirname(__file__), "images/cheetah1.jpg"),
        ]
    ]
    * 3,
    title="Kitchen Sink",
    description="Try out all the components!",
    article="Learn more about [Gradio](http://gradio.app)",
    cache_examples=True,
)

demo.launch(server_port=9876)

Traceback (most recent call last):
  File "c:\Users\a\build\gradio-test\main18.py", line 13, in <module>
    demo = gr.Interface(
  File "C:\Users\a\build\gradio-test\venv\lib\site-packages\gradio\interface.py", line 475, in __init__
    self.render_examples()
  File "C:\Users\a\build\gradio-test\venv\lib\site-packages\gradio\interface.py", line 791, in render_examples
    self.examples_handler = Examples(
  File "C:\Users\a\build\gradio-test\venv\lib\site-packages\gradio\helpers.py", line 70, in create_examples
    utils.synchronize_async(examples_obj.create)
  File "C:\Users\a\build\gradio-test\venv\lib\site-packages\gradio\utils.py", line 516, in synchronize_async
    return fsspec.asyn.sync(fsspec.asyn.get_loop(), func, *args, **kwargs)
  File "C:\Users\a\build\gradio-test\venv\lib\site-packages\fsspec\asyn.py", line 100, in sync
    raise return_result
  File "C:\Users\a\build\gradio-test\venv\lib\site-packages\fsspec\asyn.py", line 55, in _runner
    result[0] = await coro
  File "C:\Users\a\build\gradio-test\venv\lib\site-packages\gradio\helpers.py", line 277, in create
    await self.cache()
  File "C:\Users\a\build\gradio-test\venv\lib\site-packages\gradio\helpers.py", line 311, in cache
    prediction = await Context.root_block.process_api(
  File "C:\Users\a\build\gradio-test\venv\lib\site-packages\gradio\blocks.py", line 1073, in process_api
    inputs = self.preprocess_data(fn_index, inputs, state)
  File "C:\Users\a\build\gradio-test\venv\lib\site-packages\gradio\blocks.py", line 962, in preprocess_data
    processed_input.append(block.preprocess(inputs[i]))
  File "C:\Users\a\build\gradio-test\venv\lib\site-packages\gradio\components.py", line 1598, in preprocess
    assert isinstance(x, dict)
AssertionError

[catboxanon]

I think this is an even simpler example. If you upload an image, and then collapse and uncollapse the accordion, the same error occurs after clicking the button, and furthermore before even clicking the button the image disappears from the UI entirely, rendering the image component unusable.

import gradio as gr

with gr.Blocks() as demo:
    with gr.Accordion(label='test'):
        img = gr.Image(source='upload', tool='sketch')
    button = gr.Button("Print to console")
    button.click(lambda x: print(x), inputs=[img], outputs=[])

demo.launch()

This is a pretty large issue for the ControlNet extension for the Stable Diffusion web UI currently, because the interface for extensions tend be accordions that the user often collapses to save space.

[catboxanon]

I think this may have been resolved by https://github.com/gradio-app/gradio/pull/4073? I can run the demo I mentioned above just fine now.

[abidlabs]

Two different issues are at play here. The bug that was affecting your demo has been fixed, but there is still a bug affecting @space-nuko's original example

[abidlabs]

Moving to 4.0 milestone when we will rework the Image component (cc @pngwn)