search

turboderp / exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs
MIT License
3.18k stars 233 forks source link

Problem with blinker... #475

Open 0wwafa opened 1 month ago

0wwafa commented 1 month ago 
Cloning into 'exui'...
remote: Enumerating objects: 569, done.
remote: Counting objects: 100% (197/197), done.
remote: Compressing objects: 100% (99/99), done.
remote: Total 569 (delta 142), reused 134 (delta 98), pack-reused 372
Receiving objects: 100% (569/569), 2.82 MiB | 12.95 MiB/s, done.
Resolving deltas: 100% (336/336), done.
Requirement already satisfied: torch>=2.1.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 1)) (2.3.0+cu121)
Collecting pynvml (from -r requirements.txt (line 2))
  Downloading pynvml-11.5.0-py3-none-any.whl (53 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.1/53.1 kB 710.7 kB/s eta 0:00:00
Collecting exllamav2>=0.0.21 (from -r requirements.txt (line 3))
  Downloading exllamav2-0.1.1-py3-none-any.whl (245 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 245.0/245.0 kB 2.6 MB/s eta 0:00:00
Collecting Flask>=2.3.2 (from -r requirements.txt (line 4))
  Downloading flask-3.0.3-py3-none-any.whl (101 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 101.7/101.7 kB 7.0 MB/s eta 0:00:00
Collecting waitress>=2.1.2 (from -r requirements.txt (line 5))
  Downloading waitress-3.0.0-py3-none-any.whl (56 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.7/56.7 kB 4.4 MB/s eta 0:00:00
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=2.1.0->-r requirements.txt (line 1)) (3.14.0)
Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.10/dist-packages (from torch>=2.1.0->-r requirements.txt (line 1)) (4.11.0)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=2.1.0->-r requirements.txt (line 1)) (1.12)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=2.1.0->-r requirements.txt (line 1)) (3.3)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=2.1.0->-r requirements.txt (line 1)) (3.1.4)
Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch>=2.1.0->-r requirements.txt (line 1)) (2023.6.0)
Collecting nvidia-cuda-nvrtc-cu12==12.1.105 (from torch>=2.1.0->-r requirements.txt (line 1))
  Using cached nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (23.7 MB)
Collecting nvidia-cuda-runtime-cu12==12.1.105 (from torch>=2.1.0->-r requirements.txt (line 1))
  Using cached nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (823 kB)
Collecting nvidia-cuda-cupti-cu12==12.1.105 (from torch>=2.1.0->-r requirements.txt (line 1))
  Using cached nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (14.1 MB)
Collecting nvidia-cudnn-cu12==8.9.2.26 (from torch>=2.1.0->-r requirements.txt (line 1))
  Using cached nvidia_cudnn_cu12-8.9.2.26-py3-none-manylinux1_x86_64.whl (731.7 MB)
Collecting nvidia-cublas-cu12==12.1.3.1 (from torch>=2.1.0->-r requirements.txt (line 1))
  Using cached nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl (410.6 MB)
Collecting nvidia-cufft-cu12==11.0.2.54 (from torch>=2.1.0->-r requirements.txt (line 1))
  Using cached nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl (121.6 MB)
Collecting nvidia-curand-cu12==10.3.2.106 (from torch>=2.1.0->-r requirements.txt (line 1))
  Using cached nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl (56.5 MB)
Collecting nvidia-cusolver-cu12==11.4.5.107 (from torch>=2.1.0->-r requirements.txt (line 1))
  Using cached nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl (124.2 MB)
Collecting nvidia-cusparse-cu12==12.1.0.106 (from torch>=2.1.0->-r requirements.txt (line 1))
  Using cached nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl (196.0 MB)
Collecting nvidia-nccl-cu12==2.20.5 (from torch>=2.1.0->-r requirements.txt (line 1))
  Using cached nvidia_nccl_cu12-2.20.5-py3-none-manylinux2014_x86_64.whl (176.2 MB)
Collecting nvidia-nvtx-cu12==12.1.105 (from torch>=2.1.0->-r requirements.txt (line 1))
  Using cached nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl (99 kB)
Requirement already satisfied: triton==2.3.0 in /usr/local/lib/python3.10/dist-packages (from torch>=2.1.0->-r requirements.txt (line 1)) (2.3.0)
Collecting nvidia-nvjitlink-cu12 (from nvidia-cusolver-cu12==11.4.5.107->torch>=2.1.0->-r requirements.txt (line 1))
  Downloading nvidia_nvjitlink_cu12-12.5.40-py3-none-manylinux2014_x86_64.whl (21.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 21.3/21.3 MB 25.3 MB/s eta 0:00:00
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from exllamav2>=0.0.21->-r requirements.txt (line 3)) (2.0.3)
Collecting ninja (from exllamav2>=0.0.21->-r requirements.txt (line 3))
  Downloading ninja-1.11.1.1-py2.py3-none-manylinux1_x86_64.manylinux_2_5_x86_64.whl (307 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 307.2/307.2 kB 26.7 MB/s eta 0:00:00
Collecting fastparquet (from exllamav2>=0.0.21->-r requirements.txt (line 3))
  Downloading fastparquet-2024.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 76.0 MB/s eta 0:00:00
Requirement already satisfied: safetensors>=0.3.2 in /usr/local/lib/python3.10/dist-packages (from exllamav2>=0.0.21->-r requirements.txt (line 3)) (0.4.3)
Requirement already satisfied: sentencepiece>=0.1.97 in /usr/local/lib/python3.10/dist-packages (from exllamav2>=0.0.21->-r requirements.txt (line 3)) (0.1.99)
Requirement already satisfied: pygments in /usr/local/lib/python3.10/dist-packages (from exllamav2>=0.0.21->-r requirements.txt (line 3)) (2.16.1)
Collecting websockets (from exllamav2>=0.0.21->-r requirements.txt (line 3))
  Downloading websockets-12.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (130 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 130.2/130.2 kB 14.6 MB/s eta 0:00:00
Requirement already satisfied: regex in /usr/local/lib/python3.10/dist-packages (from exllamav2>=0.0.21->-r requirements.txt (line 3)) (2024.5.15)
Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from exllamav2>=0.0.21->-r requirements.txt (line 3)) (1.25.2)
Requirement already satisfied: Werkzeug>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from Flask>=2.3.2->-r requirements.txt (line 4)) (3.0.3)
Requirement already satisfied: itsdangerous>=2.1.2 in /usr/local/lib/python3.10/dist-packages (from Flask>=2.3.2->-r requirements.txt (line 4)) (2.2.0)
Requirement already satisfied: click>=8.1.3 in /usr/local/lib/python3.10/dist-packages (from Flask>=2.3.2->-r requirements.txt (line 4)) (8.1.7)
Collecting blinker>=1.6.2 (from Flask>=2.3.2->-r requirements.txt (line 4))
  Downloading blinker-1.8.2-py3-none-any.whl (9.5 kB)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=2.1.0->-r requirements.txt (line 1)) (2.1.5)
Collecting cramjam>=2.3 (from fastparquet->exllamav2>=0.0.21->-r requirements.txt (line 3))
  Downloading cramjam-2.8.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 75.6 MB/s eta 0:00:00
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from fastparquet->exllamav2>=0.0.21->-r requirements.txt (line 3)) (24.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas->exllamav2>=0.0.21->-r requirements.txt (line 3)) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->exllamav2>=0.0.21->-r requirements.txt (line 3)) (2023.4)
Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas->exllamav2>=0.0.21->-r requirements.txt (line 3)) (2024.1)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=2.1.0->-r requirements.txt (line 1)) (1.3.0)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas->exllamav2>=0.0.21->-r requirements.txt (line 3)) (1.16.0)
Installing collected packages: ninja, websockets, waitress, pynvml, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, cramjam, blinker, nvidia-cusparse-cu12, nvidia-cudnn-cu12, Flask, nvidia-cusolver-cu12, fastparquet, exllamav2
  Attempting uninstall: blinker
    Found existing installation: blinker 1.4
ERROR: Cannot uninstall 'blinker'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall
turboderp commented 1 month ago

Is this in Colab?

0wwafa commented 1 month ago

Is this in Colab?

yep.

turboderp commented 1 month ago

Honestly I have no idea what blinker is. Best bet I have is to try it with pip install --ignore-installed ... to prevent it from trying to upgrade that package, as it's apparently not managed by pip.