search

oobabooga / text-generation-webui

A Gradio web UI for Large Language Models.
GNU Affero General Public License v3.0
40.78k stars 5.33k forks source link

Error when loading multimodal extension #6052

Open RodriMora opened 6 months ago

RodriMora commented 6 months ago

Describe the bug

When I toggle the option for multimodal the software crashes.

Is there an existing issue for this?

Reproduction

Turning on the "multimodal" toggle in the "Session" tab throws this error and crashes textgen webui.

Screenshot

image image

Logs

╭───────────────────────────────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────────────────────────────────────────────────────────────╮
│ /home/ubuntu/text-generation-webui/server.py:263 in <module>                                                                                                                                                                        │
│                                                                                                                                                                                                                                     │
│   262                 time.sleep(0.5)                                                                                                                                                                                               │
│ ❱ 263                 create_interface()                                                                                                                                                                                            │
│   264                                                                                                                                                                                                                               │
│                                                                                                                                                                                                                                     │
│ /home/ubuntu/text-generation-webui/server.py:156 in create_interface                                                                                                                                                                │
│                                                                                                                                                                                                                                     │
│   155         extensions_module.create_extensions_tabs()  # Extensions tabs                                                                                                                                                         │
│ ❱ 156         extensions_module.create_extensions_block()  # Extensions block                                                                                                                                                       │
│   157                                                                                                                                                                                                                               │
│                                                                                                                                                                                                                                     │
│ /home/ubuntu/text-generation-webui/modules/extensions.py:199 in create_extensions_block                                                                                                                                             │
│                                                                                                                                                                                                                                     │
│   198                 extension, _ = row                                                                                                                                                                                            │
│ ❱ 199                 extension.ui()                                                                                                                                                                                                │
│   200                                                                                                                                                                                                                               │
│                                                                                                                                                                                                                                     │
│ /home/ubuntu/text-generation-webui/extensions/multimodal/script.py:99 in ui                                                                                                                                                         │
│                                                                                                                                                                                                                                     │
│    98     global multimodal_embedder                                                                                                                                                                                                │
│ ❱  99     multimodal_embedder = MultimodalEmbedder(params)                                                                                                                                                                          │
│   100     with gr.Column():                                                                                                                                                                                                         │
│                                                                                                                                                                                                                                     │
│ /home/ubuntu/text-generation-webui/extensions/multimodal/multimodal_embedder.py:27 in __init__                                                                                                                                      │
│                                                                                                                                                                                                                                     │
│    26     def __init__(self, params: dict):                                                                                                                                                                                         │
│ ❱  27         pipeline, source = load_pipeline(params)                                                                                                                                                                              │
│    28         self.pipeline = pipeline                                                                                                                                                                                              │
│                                                                                                                                                                                                                                     │
│ /home/ubuntu/text-generation-webui/extensions/multimodal/pipeline_loader.py:34 in load_pipeline                                                                                                                                     │
│                                                                                                                                                                                                                                     │
│   33     else:                                                                                                                                                                                                                      │
│ ❱ 34         model_name = shared.args.model.lower()                                                                                                                                                                                 │
│   35         for k in pipeline_modules:                                                                                                                                                                                             │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: 'NoneType' object has no attribute 'lower'

System Info

Running locally. Latest version as per this date, just run the update script with option A) and B)

CPU: EPYC 7402
RAM: 512GB
GPU: 4x 3090

Ubuntu 22.04.4

Nvidia driver 550.54.15
Touch-Night commented 5 months ago

You should run multimodal model with command flags, not just simply check it in Session tab.

RodriMora commented 5 months ago

You should run multimodal model with command flags, not just simply check it in Session tab.

you mean add --multimodal in the CMD_FLAGS.txt?

Touch-Night commented 5 months ago

Yes, but multimodal extension is now buggy. I failed to load wojtab_llava-7b-v0-4bit-128g model and TheBloke_vicuna-7B-1.1-GPTQ doesn't seems to work.

raffaeler commented 4 months ago

@Touch-Night When I add --multimodal in CMD_FLAGS.txt, I get the following error:

server.py: error: argument --multimodal-pipeline: expected one argument
Touch-Night commented 4 months ago

It seems broken right now, I cannot load multimodal models either.

raffaeler commented 4 months ago

Thank you for confirming @Touch-Night

daveposh commented 4 months ago

@Touch-Night When I add --multimodal in CMD_FLAGS.txt, I get the following error:

server.py: error: argument --multimodal-pipeline: expected one argument

you need to load a pipeline with the CMD FLAG example: --multimodal-pipeline llava-7b

Available pipelines ['llava-7b','llava-13b', 'llava-llama-2-13b', 'llava-v1.5-13b', 'llava-v1.5-7b']

raffaeler commented 4 months ago

Available pipelines ['llava-7b','llava-13b', 'llava-llama-2-13b', 'llava-v1.5-13b', 'llava-v1.5-7b']

what about phi-3-vision instead?

TIA

daveposh commented 4 months ago

Available pipelines ['llava-7b','llava-13b', 'llava-llama-2-13b', 'llava-v1.5-13b', 'llava-v1.5-7b']

@Touch-Night what about phi-3-vision instead?

TIA

Install them under extensions/multimodal/pipelines

Default available are: ['llava-7b','llava-13b', 'llava-llama-2-13b', 'llava-v1.5-13b', 'llava-v1.5-7b']

daveposh commented 4 months ago

Multimodal Support

Touch-Night commented 4 months ago

https://github.com/oobabooga/text-generation-webui/pull/6089