InternLM / InternLM-XComposer

InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output
2.14k stars 133 forks source link

[bugs] web_demo.py cannot perform multimodal chat (InternLM-XComposer-1.0) #161

Closed JJJYmmm closed 5 months ago

JJJYmmm commented 5 months ago

Problem

When I exec python examples/web_demo.py in directory InternLM-XComposer/InternLM-XComposer-1.0/ to launch a web demo(InternLM-XComposer-1.0) and upload a image to perform multimodal chat, bug below occured.

Traceback (most recent call last):
  File "/home/xxx/anaconda3/envs/intern_clean/lib/python3.9/site-packages/gradio/queueing.py", line 388, in call_prediction
    output = await route_utils.call_process_api(
  File "/home/xxx/anaconda3/envs/intern_clean/lib/python3.9/site-packages/gradio/route_utils.py", line 219, in call_process_api
    output = await app.get_blocks().process_api(
  File "/home/xxx/anaconda3/envs/intern_clean/lib/python3.9/site-packages/gradio/blocks.py", line 1437, in process_api
    result = await self.call_function(
  File "/home/xxx/anaconda3/envs/intern_clean/lib/python3.9/site-packages/gradio/blocks.py", line 1109, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/home/xxx/anaconda3/envs/intern_clean/lib/python3.9/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/home/xxx/anaconda3/envs/intern_clean/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 2134, in run_sync_in_worker_thread
    return await future
  File "/home/xxx/anaconda3/envs/intern_clean/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 851, in run
    result = context.run(func, *args)
  File "/home/xxx/anaconda3/envs/intern_clean/lib/python3.9/site-packages/gradio/utils.py", line 650, in wrapper
    response = f(*args, **kwargs)
  File "/home/xxx/projects/InternLM-XComposer/InternLM-XComposer-1.0/examples/web_demo.py", line 694, in chat_ask
    return (state, img_list, state.to_gradio_chatbot(), "",
  File "/home/xxx/projects/InternLM-XComposer/InternLM-XComposer-1.0/../demo_asset/conversation.py", line 97, in to_gradio_chatbot
    for j, image in enumerate(images):
TypeError: 'Image' object is not iterable

Analysis

The reason for this error is incorrect reference between directory InternLM-XComposer/demo_asset and InternLM-XComposer/InternLM-XComposer-1.0/demo_asset because in web_demo.py line 5, https://github.com/InternLM/InternLM-XComposer/blob/cee56fc4e10e677d7a896b7a9ce13259f1200a4e/InternLM-XComposer-1.0/examples/web_demo.py#L5 it add .. to system path, if I exec web demo in directory InternLM-XComposer/InternLM-XComposer-1.0/ (as the README.md says), the func to_gradio_chatbot will refer to InternLM-XComposer/demo_asset/conversation.py, which is suitble for InternLM-XComposer2 but no for InternLM-XComposer.

Solution

There are three solutions to fix the bug, each one is ok (maybe ③ is better?)

① remove web_demo.py line 5 https://github.com/InternLM/InternLM-XComposer/blob/cee56fc4e10e677d7a896b7a9ce13259f1200a4e/InternLM-XComposer-1.0/examples/web_demo.py#L5

② just correct README.md, launch web demo in sub-dir examples.

https://github.com/InternLM/InternLM-XComposer/blob/cee56fc4e10e677d7a896b7a9ce13259f1200a4e/InternLM-XComposer-1.0/README.md#L563

③ add addtional check for image or image list before https://github.com/InternLM/InternLM-XComposer/blob/cee56fc4e10e677d7a896b7a9ce13259f1200a4e/demo_asset/conversation.py#L97

if type(images) is not list:
    images = [images]