search

oobabooga / text-generation-webui

A Gradio web UI for Large Language Models.
GNU Affero General Public License v3.0
40.25k stars 5.28k forks source link

Loading Model Error #5765

Open MateuszUlan opened 6 months ago

MateuszUlan commented 6 months ago

Describe the bug

There is an error when trying load a model

Is there an existing issue for this?

Reproduction

Load model with ExLlamav2_HF Model Loader

Screenshot

obraz

Logs

Traceback (most recent call last):

File "/text-generation-webui/modules/ui_model_menu.py", line 245, in load_model_wrapper

shared.model, shared.tokenizer = load_model(selected_model, loader)

                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/text-generation-webui/modules/models.py", line 93, in load_model

output = load_func_map[loader](model_name)

         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/text-generation-webui/modules/models.py", line 384, in ExLlamav2_HF_loader

from modules.exllamav2_hf import Exllamav2HF

File "/text-generation-webui/modules/exllamav2_hf.py", line 12, in

from exllamav2 import (

ModuleNotFoundError: No module named 'exllamav2'

System Info

OS: Windows 10 Enterprise 
GPU: Nvidia A6000 AdaGeneration 

Also 

I have use Windows Docker Desktop to containerize text-generation-webui
Ctmax-ui commented 6 months ago

i am also having the same problem!!, can anyone help us out

YakuzaSuske commented 6 months ago

I think it's an installation issue but... i don't know if it will work but... here is my copy of the modules folder. Hope that helps somehow.

modules.zip

Ctmax-ui commented 6 months ago

Sigh what a drag i am trying to learn AI & ML and now this issue!!

HoustonDad commented 6 months ago

Same issue here, not sure what's wrong. I tried HEAD zip and the latest release zip, neither seem to work on Ubuntu 22.04

Downloading the model to models/TheBloke_CapybaraHermes-2.5-Mistral-7B-GPTQ
23:12:07-484568 INFO     Loading "TheBloke_CapybaraHermes-2.5-Mistral-7B-GPTQ"
23:12:07-488447 ERROR    Failed to load the model.
Traceback (most recent call last):
  File "/root/text-generation-webui-snapshot-2024-03-31/modules/ui_model_menu.py", line 245, in load_model_wrapper
    shared.model, shared.tokenizer = load_model(selected_model, loader)
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/text-generation-webui-snapshot-2024-03-31/modules/models.py", line 86, in load_model
    output = load_func_map[loader](model_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/text-generation-webui-snapshot-2024-03-31/modules/models.py", line 349, in ExLlamav2_HF_loader
    from modules.exllamav2_hf import Exllamav2HF
  File "/root/text-generation-webui-snapshot-2024-03-31/modules/exllamav2_hf.py", line 7, in <module>
    from exllamav2 import (
ModuleNotFoundError: No module named 'exllamav2'
Ctmax-ui commented 5 months ago

can any benevolent god help poor soul like us??

fluffyloopy commented 4 months ago

I had to install both requirements.txt and requirements_cpu_only.txt to use GGUF models on my system with an nvidia GPU

Ctmax-ui commented 3 months ago

the software is not working on low pc, i have i5 3 gen, 8gb ram, 256ssd, h61 board, and 720gpu should i need some upgrade to run the software??

TheKevinWang commented 2 weeks ago

same problem mac m1 max with qwen2-72b-instruct-q4_k_m.gguf git clone and ./start-macos.sh

Ctmax-ui commented 2 weeks ago

dose the developer has any solution regarding it, please help poor soul like us.