search

oobabooga / text-generation-webui

A Gradio web UI for Large Language Models.
GNU Affero General Public License v3.0
40.42k stars 5.3k forks source link

Too many problems installing on Debian 12 #5821

Closed tholonia closed 4 months ago

tholonia commented 6 months ago

Describe the bug

start_linux.sh fails

The following packages were not installed with ./start_linux.sh and had to be installed manually into the ${HOME}/src/text-generation-webui/installer_files/env conda environment.

conda install gradio

conda install pyyaml

conda install accelerate

conda install pillow

conda install markdown

conda install transformers

conda install numba

conda install datasets

conda install peft

Running server.py fails on exception (see logs included)

It would be much cleaner and easier if the installation were not a 1-click-fits-all, but rather 1) instruction to create the correct conda environment 2) instruction on installing and configuring Oobabooga 3) instructions on starting server

Is there an existing issue for this?

Reproduction

On Debian 12, follow instructions from README.md

Screenshot

No response

Logs

when running "python ./server.py"

│ /home/jw/miniforge3/lib/python3.10/site-packages/numpy/core/__init__.py:24 in <module>                                                                                                                          │
│                                                                                                                                                                                                                 │
│    23 try:                                                                                                                                                                                                      │
│ ❱  24     from . import multiarray                                                                                                                                                                              │
│    25 except ImportError as exc:                                                                                                                                                                                │
│                                                                                                                                                                                                                 │
│ /home/jw/miniforge3/lib/python3.10/site-packages/numpy/core/multiarray.py:10 in <module>                                                                                                                        │
│                                                                                                                                                                                                                 │
│      9 import functools                                                                                                                                                                                         │
│ ❱   10 from . import overrides                                                                                                                                                                                  │
│     11 from . import _multiarray_umath                                                                                                                                                                          │
│                                                                                                                                                                                                                 │
│ /home/jw/miniforge3/lib/python3.10/site-packages/numpy/core/overrides.py:8 in <module>                                                                                                                          │
│                                                                                                                                                                                                                 │
│     7 from .._utils._inspect import getargspec                                                                                                                                                                  │
│ ❱   8 from numpy.core._multiarray_umath import (                                                                                                                                                                │
│     9     add_docstring,  _get_implementing_args, _ArrayFunctionDispatcher)                                                                                                                                     │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'

During handling of the above exception, another exception occurred:

╭─────────────────────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ───────────────────────────────────────────────────────────────────────────────────────╮
│ /home/jw/miniforge3/lib/python3.10/site-packages/numpy/__init__.py:130 in <module>                                                                                                                              │
│                                                                                                                                                                                                                 │
│   129     try:                                                                                                                                                                                                  │
│ ❱ 130         from numpy.__config__ import show as show_config                                                                                                                                                  │
│   131     except ImportError as e:                                                                                                                                                                              │
│                                                                                                                                                                                                                 │
│ /home/jw/miniforge3/lib/python3.10/site-packages/numpy/__config__.py:4 in <module>                                                                                                                              │
│                                                                                                                                                                                                                 │
│     3 from enum import Enum                                                                                                                                                                                     │
│ ❱   4 from numpy.core._multiarray_umath import (                                                                                                                                                                │
│     5     __cpu_features__,                                                                                                                                                                                     │
│                                                                                                                                                                                                                 │
│ /home/jw/miniforge3/lib/python3.10/site-packages/numpy/core/__init__.py:50 in <module>                                                                                                                          │
│                                                                                                                                                                                                                 │
│    49         __version__, exc)                                                                                                                                                                                 │
│ ❱  50     raise ImportError(msg)                                                                                                                                                                                │
│    51 finally:                                                                                                                                                                                                  │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.11 from "/home/jw/store/src/text-generation-webui/installer_files/env/bin/python"
  * The NumPy version is: "1.26.4"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: No module named 'numpy.core._multiarray_umath'

The above exception was the direct cause of the following exception:

╭─────────────────────────────────────────────────────────────────────────────────────── Traceback (most recent call last) ───────────────────────────────────────────────────────────────────────────────────────╮
│ /home/jw/store/src/text-generation-webui/./server.py:6 in <module>                                                                                                                                              │
│                                                                                                                                                                                                                 │
│     5                                                                                                                                                                                                           │
│ ❱   6 import accelerate  # This early import makes Intel GPUs happy                                                                                                                                             │
│     7                                                                                                                                                                                                           │
│                                                                                                                                                                                                                 │
│ /home/jw/store/src/text-generation-webui/installer_files/env/lib/python3.11/site-packages/accelerate/__init__.py:16 in <module>                                                                                 │
│                                                                                                                                                                                                                 │
│   15                                                                                                                                                                                                            │
│ ❱ 16 from .accelerator import Accelerator                                                                                                                                                                       │
│   17 from .big_modeling import (                                                                                                                                                                                │
│                                                                                                                                                                                                                 │
│ /home/jw/store/src/text-generation-webui/installer_files/env/lib/python3.11/site-packages/accelerate/accelerator.py:32 in <module>                                                                              │
│                                                                                                                                                                                                                 │
│     31                                                                                                                                                                                                          │
│ ❱   32 import torch                                                                                                                                                                                             │
│     33 import torch.utils.hooks as hooks                                                                                                                                                                        │
│                                                                                                                                                                                                                 │
│ /home/jw/store/src/text-generation-webui/installer_files/env/lib/python3.11/site-packages/torch/__init__.py:1120 in <module>                                                                                    │
│                                                                                                                                                                                                                 │
│   1119 from ._tensor import Tensor                                                                                                                                                                              │
│ ❱ 1120 from .storage import _StorageBase, TypedStorage, _LegacyStorage, UntypedStorage, _warn_t                                                                                                                 │
│   1121                                                                                                                                                                                                          │
│                                                                                                                                                                                                                 │
│ /home/jw/store/src/text-generation-webui/installer_files/env/lib/python3.11/site-packages/torch/storage.py:14 in <module>                                                                                       │
│                                                                                                                                                                                                                 │
│     13 try:                                                                                                                                                                                                     │
│ ❱   14     import numpy as np                                                                                                                                                                                   │
│     15     HAS_NUMPY = True                                                                                                                                                                                     │
│                                                                                                                                                                                                                 │
│ /home/jw/miniforge3/lib/python3.10/site-packages/numpy/__init__.py:135 in <module>                                                                                                                              │
│                                                                                                                                                                                                                 │
│   134         your python interpreter from there."""                                                                                                                                                            │
│ ❱ 135         raise ImportError(msg) from e                                                                                                                                                                     │
│   136                                                                                                                                                                                                           │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
ImportError: Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and relaunch
        your python interpreter from there.
(/home/jw/store/src/text-generation-webui/installer_files/env) ➜  text-generation-webui git:(main) ✗

System Info

GPU
 NVIDIA-SMI 525.147.05   Driver Version: 525.147.05   CUDA Version: 12.0     |
 NVIDIA GeForce RTX 4080Ti

OS
 Debian GNU/Linux 12 
 Linux jw-box 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01) x86_64 GNU/Linux

CPU
 Architecture:                       x86_64
 CPU op-mode(s):                     32-bit, 64-bit
 Address sizes:                      39 bits physical, 48 bits virtual
 CPU(s):                             16
 Vendor ID:                          GenuineIntel
 Model name:                         Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz

MEMORY
 49172584
mmealman commented 6 months ago

Hi. I run text gen on 3 different Debian 12.5 servers. My first recommendation would be to upgrade your Nvidia driver. The 525 driver may not be your issue, but it will cause you to use more VRAM and may cause CUDA library issues since your driver only supports CUDA 12.0. See https://www.linuxcapable.com/install-nvidia-drivers-on-debian/ I've used 535, 54x and am now on 550 without issues.

My specific means for installing is:

cd src
git clone https://github.com/oobabooga/text-generation-webui
cd text-generation-webui/
./start_linux.sh --listen

I used to envelope the above in its own conda environment but that broke for installs awhile ago and I now run the above outside of any virtualized python env. My global python version is 3.11.5.

I'm also running commit 5b91dbb73bc96722314dc04479cd5a4e732fc9bc

It may be possible there was a very recent bug introduced in the last few commits. If updating your driver doesn't help, try doing a fresh install on that commit instead of what's most recent.

Alkohole commented 6 months ago

I don't know why auto installation is so attractive, I've never had a good experience with it on devices with multiple versions of python, use "Manual installation using Conda" from README.md.

PS: your version of python jumping from 3.10 to 3.11 and back again during a single process.

github-actions[bot] commented 4 months ago

This issue has been closed due to inactivity for 2 months. If you believe it is still relevant, please leave a comment below. You can tag a developer in your comment.