ParisNeo / lollms-webui

Lord of Large Language Models Web User Interface
https://lollms.com
Apache License 2.0
4.27k stars 537 forks source link

missing python package accelerate and autoawq #458

Open johanno opened 9 months ago

johanno commented 9 months ago

Expected Behavior

./linux_run.sh start with no error

Current Behavior

Failed to load model.
Returned exception: 'bool' object has no attribute 'value'
Traceback (most recent call last):
  File "/mnt/games_fast/lollms-webui/lollms-webui/lollms_core/lollms/app.py", line 246, in load_model
    model = ModelBuilder(self.binding).get_model()
  File "/mnt/games_fast/lollms-webui/lollms-webui/lollms_core/lollms/binding.py", line 584, in __init__
    self.build_model()
  File "/mnt/games_fast/lollms-webui/lollms-webui/lollms_core/lollms/binding.py", line 587, in build_model
    self.model = self.binding.build_model()
  File "/mnt/games_fast/lollms-webui/lollms-webui/zoos/bindings_zoo/hugging_face/__init__.py", line 243, in build_model
    self.model:AutoModelForCausalLM = AutoModelForCausalLM.from_pretrained(str(model_path),
  File "/mnt/games_fast/lollms-webui/installer_files/lollms_env/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
    return model_class.from_pretrained(
  File "/mnt/games_fast/lollms-webui/installer_files/lollms_env/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2863, in from_pretrained
    raise ImportError(
ImportError: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install accelerate`

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/mnt/games_fast/lollms-webui/lollms-webui/lollms_core/lollms/app.py", line 101, in __init__
    self.model          = self.load_model()
  File "/mnt/games_fast/lollms-webui/lollms-webui/lollms_core/lollms/app.py", line 251, in load_model
    self.notify("Couldn't load model.", False)
  File "/mnt/games_fast/lollms-webui/lollms-webui/api/__init__.py", line 1609, in notify
    'notification_type': notification_type.value,
AttributeError: 'bool' object has no attribute 'value'

after installing accelerate: ....

raise ImportError("Loading an AWQ quantized model requires auto-awq library (`pip install autoawq`)")
ImportError: Loading an AWQ quantized model requires auto-awq library (`pip install autoawq`)

Steps to Reproduce

./linux_run.sh on debian trixie

Possible Solution

installer_files/lollms_env/bin/python -m pip install accelerate

installer_files/lollms_env/bin/python -m pip install autoawq

Context

Please provide any additional context about the issue.

Screenshots

If applicable, add screenshots to help explain the issue.

johanno commented 9 months ago

seems only to come up once you have installed some models

ParisNeo commented 9 months ago

Sorry, I just fixed it. It was the move to the new notification system on lollms that caused this. I forgot to migrate this code to the new format Thanks for the message