search

oobabooga / text-generation-webui

A Gradio web UI for Large Language Models.
GNU Affero General Public License v3.0
40.84k stars 5.34k forks source link

Can't load models on fresh install, cpu mode. #4675

Closed Ranko-Saotome closed 10 months ago

Ranko-Saotome commented 1 year ago

Describe the bug

So I've recently reinstalled the text generation UI using the one click start_linux.sh script. My install was working earlier but I went to do an update and it messed up so I had to reinstall. but now whenever try to load models I keep getting connection errors and then I find the script seems to crash with an illegal instruction.

Is there an existing issue for this?

Reproduction

i Don't really Know

Screenshot

No response

Logs

2023-11-20 02:06:56 INFO:Loading mythomax-l2-13b.Q5_K_S.gguf...
2023-11-20 02:06:56 INFO:llama.cpp weights detected: models/mythomax-l2-13b.Q5_K_S.gguf
2023-11-20 02:06:56 INFO:Cache capacity is 0 bytes
Illegal instruction

System Info

        `osssssssssssssssssssso`           owner@Owner-Game-PC 
       .osssssssssssssssssssssso.          ------------------- 
      .+oooooooooooooooooooooooo+.         OS: Zorin OS 16.3 x86_64 
                                           Host: MS-7921 1.0 
                                           Kernel: 5.15.0-84-generic 
  `::::::::::::::::::::::.         .:`     Uptime: 28 mins 
 `+ssssssssssssssssss+:.`     `.:+ssso`    Packages: 2114 (dpkg), 69 (flatpak), 
.ossssssssssssssso/.       `-+ossssssso.   Shell: zsh 5.8 
ssssssssssssso/-`      `-/osssssssssssss   Resolution: 1920x1080 
.ossssssso/-`      .-/ossssssssssssssso.   DE: GNOME 
 `+sss+:.      `.:+ssssssssssssssssss+`    WM: Mutter 
  `:.         .::::::::::::::::::::::`     WM Theme: ZorinBlue-Dark 
                                           Theme: ZorinBlue-Dark [GTK2/3] 
                                           Icons: ZorinBlue-Dark [GTK2/3] 
      .+oooooooooooooooooooooooo+.         Terminal: gnome-terminal 
       -osssssssssssssssssssssso-          CPU: Intel i5-4690K (4) @ 3.900GHz 
        `osssssssssssssssssssso`           GPU: NVIDIA GeForce GTX 770 
                                           Memory: 3019MiB / 32038MiB
danielw97 commented 1 year ago

Just to say that I'm also experiencing this issue after pulling the commits murged yesterday. Apologies I can't pinpoint an exact commit, although all I can say is that it was working prior to those commits. If there's anything that I can do to troubleshoot this let me know.

oobabooga commented 1 year ago

This should be fixed in the dev branch now.

Ranko-Saotome commented 1 year ago

I am still having the issue. im on the dev branch.

Ranko-Saotome commented 1 year ago

(/home/owner/Downloads/text-generation-webui-dev/installer_files/env) owner@Owner-Game-PC:~/Downloads/text-generation-webui-dev$ pip list Package Version

absl-py 2.0.0 accelerate 0.24.1 aiofiles 23.2.1 aiohttp 3.9.0 aiosignal 1.3.1 altair 5.1.2 antlr4-python3-runtime 4.9.3 anyio 3.7.1 appdirs 1.4.4 asttokens 2.4.1 attrs 23.1.0 beautifulsoup4 4.12.2 bitsandbytes 0.41.1 blinker 1.7.0 cachetools 5.3.2 certifi 2022.12.7 cffi 1.16.0 charset-normalizer 2.1.1 click 8.1.7 colorama 0.4.6 coloredlogs 15.0.1 contourpy 1.2.0 cramjam 2.7.0 cycler 0.12.1 datasets 2.15.0 decorator 5.1.1 deep-translator 1.9.2 dill 0.3.7 diskcache 5.6.3 docker-pycreds 0.4.0 docopt 0.6.2 einops 0.7.0 elevenlabs 0.2.24 executing 2.0.1 exllamav2 0.0.8 fastapi 0.104.1 fastparquet 2023.10.1 ffmpeg 1.4 ffmpy 0.3.1 filelock 3.9.0 Flask 3.0.0 flask-cloudflared 0.0.14 fonttools 4.45.0 frozenlist 1.4.0 fsspec 2023.10.0 gitdb 4.0.11 GitPython 3.1.40 google-auth 2.23.4 google-auth-oauthlib 1.1.0 gradio 3.50.2 gradio_client 0.6.1 grpcio 1.59.3 h11 0.14.0 httpcore 1.0.2 httpx 0.25.1 huggingface-hub 0.19.4 humanfriendly 10.0 idna 3.4 importlib-resources 6.1.1 ipython 8.17.2 itsdangerous 2.1.2 jedi 0.19.1 Jinja2 3.1.2 jsonschema 4.20.0 jsonschema-specifications 2023.11.1 kiwisolver 1.4.5 llama_cpp_python 0.2.18 llvmlite 0.41.1 Markdown 3.5.1 MarkupSafe 2.1.3 matplotlib 3.8.2 matplotlib-inline 0.1.6 more-itertools 10.1.0 mpmath 1.3.0 multidict 6.0.4 multiprocess 0.70.15 networkx 3.0 ngrok 0.12.0 ninja 1.11.1.1 num2words 0.5.13 numba 0.58.1 numpy 1.24.4 oauthlib 3.2.2 omegaconf 2.3.0 openai-whisper 20230918 optimum 1.14.0 orjson 3.9.10 packaging 23.2 pandas 2.1.3 parso 0.8.3 peft 0.6.2 pexpect 4.8.0 Pillow 10.1.0 pip 23.3 prompt-toolkit 3.0.41 protobuf 4.23.4 psutil 5.9.6 ptyprocess 0.7.0 pure-eval 0.2.2 py-cpuinfo 9.0.0 pyarrow 14.0.1 pyarrow-hotfix 0.5 pyasn1 0.5.0 pyasn1-modules 0.3.0 pycparser 2.21 pydantic 1.10.13 pydub 0.25.1 Pygments 2.17.1 pyparsing 3.1.1 python-dateutil 2.8.2 python-multipart 0.0.6 pytz 2023.3.post1 PyYAML 6.0.1 referencing 0.31.0 regex 2023.10.3 requests 2.31.0 requests-oauthlib 1.3.1 rpds-py 0.13.1 rsa 4.9 safetensors 0.4.0 scipy 1.11.4 semantic-version 2.10.0 sentencepiece 0.1.99 sentry-sdk 1.35.0 setproctitle 1.3.3 setuptools 68.0.0 six 1.16.0 smmap 5.0.1 sniffio 1.3.0 soundfile 0.12.1 soupsieve 2.5 SpeechRecognition 3.10.0 sse-starlette 1.6.5 stack-data 0.6.3 starlette 0.27.0 sympy 1.12 tensorboard 2.15.1 tensorboard-data-server 0.7.2 tiktoken 0.5.1 tokenizers 0.15.0 toolz 0.12.0 torch 2.1.1+cpu torch-grammar 0.3.3 torchaudio 2.1.1+cpu torchvision 0.16.1+cpu tqdm 4.66.1 traitlets 5.13.0 transformers 4.35.2 typing_extensions 4.8.0 tzdata 2023.3 urllib3 1.26.13 uvicorn 0.24.0.post1 wandb 0.16.0 wcwidth 0.2.10 websockets 11.0.3 Werkzeug 3.0.1 wheel 0.41.2 xxhash 3.4.1 yarl 1.9.2

Ranko-Saotome commented 1 year ago

11/site-packages/bitsandbytes/cextension.py:34: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable. warn("The installed version of bitsandbytes was compiled without GPU support. " /home/owner/Downloads/text-generation-webui-dev/installer_files/env/lib/python3.11/site-packages/bitsandbytes/libbitsandbytes_cpu.so: undefined symbol: cadam32bit_grad_fp32

Ranko-Saotome commented 1 year ago

2023-11-20 14:27:54 INFO:Loading mythomax-l2-13b.Q5_K_S.gguf... 2023-11-20 14:27:54 INFO:llama.cpp weights detected: models/mythomax-l2-13b.Q5_K_S.gguf 2023-11-20 14:27:54 INFO:Cache capacity is 0 bytes Illegal instruction

oobabooga commented 1 year ago

@Ranko-Saotome make sure to pip install -r requirements.txt after switching to the dev branch. I reproduced your issue and it was gone after using the updated wheels.

danielw97 commented 1 year ago

Just to say that this is now fixed for me when using the dev branch. Thanks for getting this done so quickly.

Ranko-Saotome commented 1 year ago

@Ranko-Saotome make sure to pip install -r requirements.txt after switching to the dev branch. I reproduced your issue and it was gone after using the updated wheels.

I downloaded the zip file for the latest Dev release I found out why I'm having issues it's because for some reason it's still got the old Wheels I will try to redownload the dev release and redo it and see if it messes up again again I did not install manually

Ranko-Saotome commented 1 year ago

@Ranko-Saotome make sure to pip install -r requirements.txt after switching to the dev branch. I reproduced your issue and it was gone after using the updated wheels.

I downloaded the zip file for the latest Dev release I found out why I'm having issues it's because for some reason it's still got the old Wheels I will try to redownload the dev release and redo it and see if it messes up again again I did not install manually

Ranko-Saotome commented 1 year ago

Thank you all for the help apparently github was giving me a zip file that still had the old requirements text even after I switched to the dev branch so I eventually just had to git clone the entire repo and do it that way.

github-actions[bot] commented 10 months ago

This issue has been closed due to inactivity for 6 weeks. If you believe it is still relevant, please leave a comment below. You can tag a developer in your comment.