search

microsoft / SoM

Set-of-Mark Prompting for GPT-4V and LMMs
MIT License
1.2k stars 96 forks source link

Implement gradio client (work in progress) #21

Closed abrichr closed 9 months ago

abrichr commented 9 months ago

There appears to be an issue using gradio_client (any version) with the original gradio version 3.42.0:

client.py:

from gradio_client import Client

client = Client("http://3.84.210.111:6092/")
result = client.predict(
    # str (filepath or URL to image) in 'Input' Image component
    "https://raw.githubusercontent.com/gradio-app/gradio/main/test/test_files/bus.png",
    # int | float (numeric value between 1 and 3) in 'Granularity' Slider component
    1,
    # str in 'Segmentation Mode' Radio component
    "Automatic",
    # int | float (numeric value between 0 and 1) in 'Mask Alpha' Slider component
    0,
    # str in 'Mark Mode' Radio component
    "Number",
    # List[str] in 'Annotation Mode' Checkboxgroup component
    ["Mask"],
    fn_index=2
)
print(result)
(venv) abrichr@MacBook-Pro-4 SoM % python client.py
Loaded as API: http://3.84.210.111:6092/ ✔
Traceback (most recent call last):
  File "/Users/abrichr/oa/src/SoM/client.py", line 4, in <module>
    result = client.predict(
  File "/Users/abrichr/oa/src/SoM/venv/lib/python3.10/site-packages/gradio_client/client.py", line 392, in predict
    return self.submit(*args, api_name=api_name, fn_index=fn_index).result()
  File "/Users/abrichr/oa/src/SoM/venv/lib/python3.10/site-packages/gradio_client/client.py", line 1578, in result
    return super().result(timeout=timeout)
  File "/usr/local/Cellar/python@3.10/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 458, in result
    return self.__get_result()
  File "/usr/local/Cellar/python@3.10/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/usr/local/Cellar/python@3.10/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/abrichr/oa/src/SoM/venv/lib/python3.10/site-packages/gradio_client/client.py", line 1250, in _inner
    predictions = _predict(*data)
  File "/Users/abrichr/oa/src/SoM/venv/lib/python3.10/site-packages/gradio_client/client.py", line 1280, in _predict
    raise ValueError(result["error"])
ValueError: None

server:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/gradio/routes.py", line 523, in run_predict
    output = await app.get_blocks().process_api(
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1435, in process_api
    inputs = self.preprocess_data(fn_index, inputs, state)
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 1245, in preprocess_data
    processed_input.append(block.preprocess(inputs[i]))
  File "/usr/src/app/./demo_som.py", line 135, in preprocess
    return super().preprocess(x)
  File "/usr/local/lib/python3.10/dist-packages/gradio/components/image.py", line 270, in preprocess
    assert isinstance(x, dict)
AssertionError

Modifying client.py to pass a dict instead:

result = client.predict(
    {
        "image": "https://raw.githubusercontent.com/gradio-app/gradio/main/test/test_files/bus.png",
    },
...
(venv) abrichr@MacBook-Pro-4 SoM % python client.py
Loaded as API: http://3.84.210.111:6092/ ✔
Traceback (most recent call last):
  File "/Users/abrichr/oa/src/SoM/client.py", line 4, in <module>
    result = client.predict(
  File "/Users/abrichr/oa/src/SoM/venv/lib/python3.10/site-packages/gradio_client/client.py", line 392, in predict
    return self.submit(*args, api_name=api_name, fn_index=fn_index).result()
  File "/Users/abrichr/oa/src/SoM/venv/lib/python3.10/site-packages/gradio_client/client.py", line 1578, in result
    return super().result(timeout=timeout)
  File "/usr/local/Cellar/python@3.10/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/usr/local/Cellar/python@3.10/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/usr/local/Cellar/python@3.10/3.10.11/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/abrichr/oa/src/SoM/venv/lib/python3.10/site-packages/gradio_client/client.py", line 1249, in _inner
    data = self.serialize(*data)
  File "/Users/abrichr/oa/src/SoM/venv/lib/python3.10/site-packages/gradio_client/client.py", line 1417, in serialize
    o = tuple([s.serialize(d) for s, d in zip(self.serializers, data)])
  File "/Users/abrichr/oa/src/SoM/venv/lib/python3.10/site-packages/gradio_client/client.py", line 1417, in <listcomp>
    o = tuple([s.serialize(d) for s, d in zip(self.serializers, data)])
  File "/Users/abrichr/oa/src/SoM/venv/lib/python3.10/site-packages/gradio_client/serializing.py", line 185, in serialize
    if utils.is_http_url_like(x):
  File "/Users/abrichr/oa/src/SoM/venv/lib/python3.10/site-packages/gradio_client/utils.py", line 251, in is_http_url_like
    return possible_url.startswith(("http://", "https://"))
AttributeError: 'dict' object has no attribute 'startswith'

Related:

https://github.com/gradio-app/gradio/issues/3623

https://github.com/gradio-app/gradio/pull/6169

https://github.com/gradio-app/gradio/issues/6355

Upgrading to gradio==4.16.0 produces a few errors in the browser, in which the UI does not finish loading:

GET https://gradio.s3-us-west-2.amazonaws.com/assets/index-20519b4d.css net::ERR_ABORTED 403 (Forbidden)
...
image

Related:

https://github.com/gradio-app/gradio/issues/6824

abrichr commented 9 months ago

Moved to https://github.com/microsoft/SoM/issues/22