search

Troyanovsky / Local-LLM-Comparison-Colab-UI

Compare the performance of different LLM that can be deployed locally on consumer hardware. Run yourself with Colab WebUI.
979 stars 142 forks source link

CUDA driver version is insufficient for CUDA runtime version #12

Closed LOKSTED closed 9 months ago

LOKSTED commented 9 months ago

This only happen with the WizardLM-1.0-Uncensored-Llama2-13B on Google colab Thank you for taking a look

/content
Selecting previously unselected package libc-ares2:amd64.
(Reading database ... 121730 files and directories currently installed.)
Preparing to unpack .../libc-ares2_1.18.1-1ubuntu0.22.04.2_amd64.deb ...
Unpacking libc-ares2:amd64 (1.18.1-1ubuntu0.22.04.2) ...
Selecting previously unselected package libaria2-0:amd64.
Preparing to unpack .../libaria2-0_1.36.0-1_amd64.deb ...
Unpacking libaria2-0:amd64 (1.36.0-1) ...
Selecting previously unselected package aria2.
Preparing to unpack .../aria2_1.36.0-1_amd64.deb ...
Unpacking aria2 (1.36.0-1) ...
Setting up libc-ares2:amd64 (1.18.1-1ubuntu0.22.04.2) ...
Setting up libaria2-0:amd64 (1.36.0-1) ...
Setting up aria2 (1.36.0-1) ...
Processing triggers for man-db (2.10.2-1) ...
Processing triggers for libc-bin (2.35-0ubuntu3.4) ...
/sbin/ldconfig.real: /usr/local/lib/libtbbbind.so.3 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtbb.so.12 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtbbbind_2_0.so.3 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtbbmalloc.so.2 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtbbbind_2_5.so.3 is not a symbolic link

/sbin/ldconfig.real: /usr/local/lib/libtbbmalloc_proxy.so.2 is not a symbolic link

Cloning into 'text-generation-webui'...
remote: Enumerating objects: 15569, done.
remote: Counting objects: 100% (15569/15569), done.
remote: Compressing objects: 100% (4561/4561), done.
remote: Total 15569 (delta 10901), reused 15433 (delta 10845), pack-reused 0
Receiving objects: 100% (15569/15569), 25.96 MiB | 24.30 MiB/s, done.
Resolving deltas: 100% (10901/10901), done.
Note: switching to 'ebb4f22028316821c3c14ad91b45e01d309f1ff0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

/content/text-generation-webui
Collecting git+https://github.com/huggingface/peft@96c0277a1b9a381b10ab34dbf84917f9b3b992e6 (from -r requirements.txt (line 21))
  Cloning https://github.com/huggingface/peft (to revision 96c0277a1b9a381b10ab34dbf84917f9b3b992e6) to /tmp/pip-req-build-7fre6n6f
  Running command git clone --filter=blob:none --quiet https://github.com/huggingface/peft /tmp/pip-req-build-7fre6n6f
  Running command git rev-parse -q --verify 'sha^96c0277a1b9a381b10ab34dbf84917f9b3b992e6'
  Running command git fetch -q https://github.com/huggingface/peft 96c0277a1b9a381b10ab34dbf84917f9b3b992e6
  Running command git checkout -q 96c0277a1b9a381b10ab34dbf84917f9b3b992e6
  Resolved https://github.com/huggingface/peft to commit 96c0277a1b9a381b10ab34dbf84917f9b3b992e6
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Ignoring bitsandbytes: markers 'platform_system == "Windows"' don't match your environment
Ignoring auto-gptq: markers 'platform_system == "Windows"' don't match your environment
Collecting auto-gptq==0.3.0+cu117 (from -r requirements.txt (line 25))
  Downloading https://github.com/PanQiWei/AutoGPTQ/releases/download/v0.3.0/auto_gptq-0.3.0+cu117-cp310-cp310-linux_x86_64.whl (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 8.6 MB/s eta 0:00:00
Ignoring exllama: markers 'platform_system == "Windows"' don't match your environment
Collecting exllama==0.0.9+cu117 (from -r requirements.txt (line 27))
  Downloading https://github.com/jllllll/exllama/releases/download/0.0.9/exllama-0.0.9+cu117-cp310-cp310-linux_x86_64.whl (355 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 355.2/355.2 kB 6.7 MB/s eta 0:00:00
Ignoring llama-cpp-python: markers 'platform_system == "Windows"' don't match your environment
Ignoring llama-cpp-python-cuda: markers 'platform_system == "Windows"' don't match your environment
Collecting llama-cpp-python-cuda==0.1.77+cu117 (from -r requirements.txt (line 33))
  Downloading https://github.com/jllllll/llama-cpp-python-cuBLAS-wheels/releases/download/textgen-webui/llama_cpp_python_cuda-0.1.77+cu117-cp310-cp310-linux_x86_64.whl (2.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.9/2.9 MB 16.9 MB/s eta 0:00:00
Collecting accelerate==0.21.0 (from -r requirements.txt (line 1))
  Downloading accelerate-0.21.0-py3-none-any.whl (244 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 244.2/244.2 kB 4.9 MB/s eta 0:00:00
Collecting colorama (from -r requirements.txt (line 2))
  Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Collecting datasets (from -r requirements.txt (line 3))
  Downloading datasets-2.16.1-py3-none-any.whl (507 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 507.1/507.1 kB 10.5 MB/s eta 0:00:00
Collecting einops (from -r requirements.txt (line 4))
  Downloading einops-0.7.0-py3-none-any.whl (44 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 44.6/44.6 kB 7.7 MB/s eta 0:00:00
Collecting fastapi==0.95.2 (from -r requirements.txt (line 5))
  Downloading fastapi-0.95.2-py3-none-any.whl (56 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.0/57.0 kB 6.8 MB/s eta 0:00:00
Collecting gradio_client==0.2.5 (from -r requirements.txt (line 6))
  Downloading gradio_client-0.2.5-py3-none-any.whl (288 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 288.1/288.1 kB 10.5 MB/s eta 0:00:00
Collecting gradio==3.33.1 (from -r requirements.txt (line 7))
  Downloading gradio-3.33.1-py3-none-any.whl (20.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20.0/20.0 MB 39.0 MB/s eta 0:00:00
Requirement already satisfied: markdown in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 8)) (3.5.2)
Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 9)) (1.23.5)
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 10)) (1.5.3)
Collecting Pillow>=9.5.0 (from -r requirements.txt (line 11))
  Downloading pillow-10.2.0-cp310-cp310-manylinux_2_28_x86_64.whl (4.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.5/4.5 MB 60.9 MB/s eta 0:00:00
Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 12)) (6.0.1)
Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 13)) (2.31.0)
Collecting safetensors==0.3.1 (from -r requirements.txt (line 14))
  Downloading safetensors-0.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 74.5 MB/s eta 0:00:00
Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 15)) (1.11.4)
Requirement already satisfied: sentencepiece in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 16)) (0.1.99)
Requirement already satisfied: tensorboard in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 17)) (2.15.1)
Collecting transformers==4.31.* (from -r requirements.txt (line 18))
  Downloading transformers-4.31.0-py3-none-any.whl (7.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.4/7.4 MB 72.7 MB/s eta 0:00:00
Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 19)) (4.66.1)
Collecting wandb (from -r requirements.txt (line 20))
  Downloading wandb-0.16.2-py3-none-any.whl (2.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 74.0 MB/s eta 0:00:00
Collecting bitsandbytes==0.41.0 (from -r requirements.txt (line 22))
  Downloading bitsandbytes-0.41.0-py3-none-any.whl (92.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 92.6/92.6 MB 9.3 MB/s eta 0:00:00
Collecting llama-cpp-python==0.1.77 (from -r requirements.txt (line 29))
  Downloading llama_cpp_python-0.1.77.tar.gz (1.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 19.1 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from accelerate==0.21.0->-r requirements.txt (line 1)) (23.2)
Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate==0.21.0->-r requirements.txt (line 1)) (5.9.5)
Requirement already satisfied: torch>=1.10.0 in /usr/local/lib/python3.10/dist-packages (from accelerate==0.21.0->-r requirements.txt (line 1)) (2.1.0+cu121)
Requirement already satisfied: pydantic!=1.7,!=1.7.1,!=1.7.2,!=1.7.3,!=1.8,!=1.8.1,<2.0.0,>=1.6.2 in /usr/local/lib/python3.10/dist-packages (from fastapi==0.95.2->-r requirements.txt (line 5)) (1.10.14)
Collecting starlette<0.28.0,>=0.27.0 (from fastapi==0.95.2->-r requirements.txt (line 5))
  Downloading starlette-0.27.0-py3-none-any.whl (66 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.0/67.0 kB 8.1 MB/s eta 0:00:00
Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from gradio_client==0.2.5->-r requirements.txt (line 6)) (2023.6.0)
Collecting httpx (from gradio_client==0.2.5->-r requirements.txt (line 6))
  Downloading httpx-0.26.0-py3-none-any.whl (75 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.9/75.9 kB 10.1 MB/s eta 0:00:00
Requirement already satisfied: huggingface-hub>=0.13.0 in /usr/local/lib/python3.10/dist-packages (from gradio_client==0.2.5->-r requirements.txt (line 6)) (0.20.3)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from gradio_client==0.2.5->-r requirements.txt (line 6)) (4.5.0)
Collecting websockets (from gradio_client==0.2.5->-r requirements.txt (line 6))
  Downloading websockets-12.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (130 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 130.2/130.2 kB 17.7 MB/s eta 0:00:00
Collecting aiofiles (from gradio==3.33.1->-r requirements.txt (line 7))
  Downloading aiofiles-23.2.1-py3-none-any.whl (15 kB)
Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from gradio==3.33.1->-r requirements.txt (line 7)) (3.9.3)
Requirement already satisfied: altair>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.33.1->-r requirements.txt (line 7)) (4.2.2)
Collecting ffmpy (from gradio==3.33.1->-r requirements.txt (line 7))
  Downloading ffmpy-0.3.1.tar.gz (5.5 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from gradio==3.33.1->-r requirements.txt (line 7)) (3.1.3)
Requirement already satisfied: markdown-it-py[linkify]>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.33.1->-r requirements.txt (line 7)) (3.0.0)
Requirement already satisfied: markupsafe in /usr/local/lib/python3.10/dist-packages (from gradio==3.33.1->-r requirements.txt (line 7)) (2.1.4)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from gradio==3.33.1->-r requirements.txt (line 7)) (3.7.1)
Collecting mdit-py-plugins<=0.3.3 (from gradio==3.33.1->-r requirements.txt (line 7))
  Downloading mdit_py_plugins-0.3.3-py3-none-any.whl (50 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.5/50.5 kB 937.9 kB/s eta 0:00:00
Collecting orjson (from gradio==3.33.1->-r requirements.txt (line 7))
  Downloading orjson-3.9.12-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (139 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 139.8/139.8 kB 20.5 MB/s eta 0:00:00
Collecting pydub (from gradio==3.33.1->-r requirements.txt (line 7))
  Downloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)
Requirement already satisfied: pygments>=2.12.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.33.1->-r requirements.txt (line 7)) (2.16.1)
Collecting python-multipart (from gradio==3.33.1->-r requirements.txt (line 7))
  Downloading python_multipart-0.0.6-py3-none-any.whl (45 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.7/45.7 kB 6.8 MB/s eta 0:00:00
Collecting semantic-version (from gradio==3.33.1->-r requirements.txt (line 7))
  Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)
Collecting uvicorn>=0.14.0 (from gradio==3.33.1->-r requirements.txt (line 7))
  Downloading uvicorn-0.27.0.post1-py3-none-any.whl (60 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.7/60.7 kB 9.7 MB/s eta 0:00:00
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.*->-r requirements.txt (line 18)) (3.13.1)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.31.*->-r requirements.txt (line 18)) (2023.12.25)
Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers==4.31.*->-r requirements.txt (line 18))
  Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.8/7.8 MB 55.0 MB/s eta 0:00:00
Requirement already satisfied: diskcache>=5.6.1 in /usr/local/lib/python3.10/dist-packages (from llama-cpp-python==0.1.77->-r requirements.txt (line 29)) (5.6.3)
Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets->-r requirements.txt (line 3)) (10.0.1)
Requirement already satisfied: pyarrow-hotfix in /usr/local/lib/python3.10/dist-packages (from datasets->-r requirements.txt (line 3)) (0.6)
Collecting dill<0.3.8,>=0.3.0 (from datasets->-r requirements.txt (line 3))
  Downloading dill-0.3.7-py3-none-any.whl (115 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 115.3/115.3 kB 17.9 MB/s eta 0:00:00
Requirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from datasets->-r requirements.txt (line 3)) (3.4.1)
Collecting multiprocess (from datasets->-r requirements.txt (line 3))
  Downloading multiprocess-0.70.16-py310-none-any.whl (134 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 134.8/134.8 kB 19.9 MB/s eta 0:00:00
Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->-r requirements.txt (line 10)) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->-r requirements.txt (line 10)) (2023.4)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->-r requirements.txt (line 13)) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->-r requirements.txt (line 13)) (3.6)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->-r requirements.txt (line 13)) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->-r requirements.txt (line 13)) (2023.11.17)
Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 17)) (1.4.0)
Requirement already satisfied: grpcio>=1.48.2 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 17)) (1.60.0)
Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 17)) (2.17.3)
Requirement already satisfied: google-auth-oauthlib<2,>=0.5 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 17)) (1.2.0)
Requirement already satisfied: protobuf<4.24,>=3.19.6 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 17)) (3.20.3)
Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 17)) (67.7.2)
Requirement already satisfied: six>1.9 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 17)) (1.16.0)
Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 17)) (0.7.2)
Requirement already satisfied: werkzeug>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from tensorboard->-r requirements.txt (line 17)) (3.0.1)
Requirement already satisfied: Click!=8.0.0,>=7.1 in /usr/local/lib/python3.10/dist-packages (from wandb->-r requirements.txt (line 20)) (8.1.7)
Collecting GitPython!=3.1.29,>=1.0.0 (from wandb->-r requirements.txt (line 20))
  Downloading GitPython-3.1.41-py3-none-any.whl (196 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 196.4/196.4 kB 26.6 MB/s eta 0:00:00
Collecting sentry-sdk>=1.0.0 (from wandb->-r requirements.txt (line 20))
  Downloading sentry_sdk-1.40.0-py2.py3-none-any.whl (257 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 257.5/257.5 kB 24.9 MB/s eta 0:00:00
Collecting docker-pycreds>=0.4.0 (from wandb->-r requirements.txt (line 20))
  Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
Collecting setproctitle (from wandb->-r requirements.txt (line 20))
  Downloading setproctitle-1.3.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (30 kB)
Requirement already satisfied: appdirs>=1.4.3 in /usr/local/lib/python3.10/dist-packages (from wandb->-r requirements.txt (line 20)) (1.4.4)
Collecting rouge (from auto-gptq==0.3.0+cu117->-r requirements.txt (line 25))
  Downloading rouge-1.0.1-py3-none-any.whl (13 kB)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio==3.33.1->-r requirements.txt (line 7)) (0.4)
Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio==3.33.1->-r requirements.txt (line 7)) (4.19.2)
Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio==3.33.1->-r requirements.txt (line 7)) (0.12.1)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.33.1->-r requirements.txt (line 7)) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.33.1->-r requirements.txt (line 7)) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.33.1->-r requirements.txt (line 7)) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.33.1->-r requirements.txt (line 7)) (6.0.4)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.33.1->-r requirements.txt (line 7)) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.33.1->-r requirements.txt (line 7)) (4.0.3)
Collecting gitdb<5,>=4.0.1 (from GitPython!=3.1.29,>=1.0.0->wandb->-r requirements.txt (line 20))
  Downloading gitdb-4.0.11-py3-none-any.whl (62 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.7/62.7 kB 10.9 MB/s eta 0:00:00
Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard->-r requirements.txt (line 17)) (5.3.2)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard->-r requirements.txt (line 17)) (0.3.0)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard->-r requirements.txt (line 17)) (4.9)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from google-auth-oauthlib<2,>=0.5->tensorboard->-r requirements.txt (line 17)) (1.3.1)
Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio==3.33.1->-r requirements.txt (line 7)) (0.1.2)
Requirement already satisfied: linkify-it-py<3,>=1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio==3.33.1->-r requirements.txt (line 7)) (2.0.2)
INFO: pip is looking at multiple versions of mdit-py-plugins to determine which version is compatible with other requirements. This could take a while.
Collecting mdit-py-plugins<=0.3.3 (from gradio==3.33.1->-r requirements.txt (line 7))
  Downloading mdit_py_plugins-0.3.2-py3-none-any.whl (50 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.4/50.4 kB 1.4 MB/s eta 0:00:00
  Downloading mdit_py_plugins-0.3.1-py3-none-any.whl (46 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 46.5/46.5 kB 5.9 MB/s eta 0:00:00
  Downloading mdit_py_plugins-0.3.0-py3-none-any.whl (43 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 43.7/43.7 kB 6.7 MB/s eta 0:00:00
  Downloading mdit_py_plugins-0.2.8-py3-none-any.whl (41 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.0/41.0 kB 6.5 MB/s eta 0:00:00
  Downloading mdit_py_plugins-0.2.7-py3-none-any.whl (41 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.0/41.0 kB 6.1 MB/s eta 0:00:00
  Downloading mdit_py_plugins-0.2.6-py3-none-any.whl (39 kB)
  Downloading mdit_py_plugins-0.2.5-py3-none-any.whl (39 kB)
INFO: pip is looking at multiple versions of mdit-py-plugins to determine which version is compatible with other requirements. This could take a while.
  Downloading mdit_py_plugins-0.2.4-py3-none-any.whl (39 kB)
  Downloading mdit_py_plugins-0.2.3-py3-none-any.whl (39 kB)
  Downloading mdit_py_plugins-0.2.2-py3-none-any.whl (39 kB)
  Downloading mdit_py_plugins-0.2.1-py3-none-any.whl (38 kB)
  Downloading mdit_py_plugins-0.2.0-py3-none-any.whl (38 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
  Downloading mdit_py_plugins-0.1.0-py3-none-any.whl (37 kB)
Collecting markdown-it-py[linkify]>=2.0.0 (from gradio==3.33.1->-r requirements.txt (line 7))
  Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 87.5/87.5 kB 14.7 MB/s eta 0:00:00
  Downloading markdown_it_py-2.2.0-py3-none-any.whl (84 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 84.5/84.5 kB 13.1 MB/s eta 0:00:00
Requirement already satisfied: anyio<5,>=3.4.0 in /usr/local/lib/python3.10/dist-packages (from starlette<0.28.0,>=0.27.0->fastapi==0.95.2->-r requirements.txt (line 5)) (3.7.1)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 1)) (1.12)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 1)) (3.2.1)
Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 1)) (2.1.0)
Collecting h11>=0.8 (from uvicorn>=0.14.0->gradio==3.33.1->-r requirements.txt (line 7))
  Downloading h11-0.14.0-py3-none-any.whl (58 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 9.5 MB/s eta 0:00:00
Collecting httpcore==1.* (from httpx->gradio_client==0.2.5->-r requirements.txt (line 6))
  Downloading httpcore-1.0.2-py3-none-any.whl (76 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 76.9/76.9 kB 11.8 MB/s eta 0:00:00
Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio_client==0.2.5->-r requirements.txt (line 6)) (1.3.0)
Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.33.1->-r requirements.txt (line 7)) (1.2.0)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.33.1->-r requirements.txt (line 7)) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.33.1->-r requirements.txt (line 7)) (4.47.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.33.1->-r requirements.txt (line 7)) (1.4.5)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.33.1->-r requirements.txt (line 7)) (3.1.1)
INFO: pip is looking at multiple versions of multiprocess to determine which version is compatible with other requirements. This could take a while.
Collecting multiprocess (from datasets->-r requirements.txt (line 3))
  Downloading multiprocess-0.70.15-py310-none-any.whl (134 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 134.8/134.8 kB 20.8 MB/s eta 0:00:00
Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.4.0->starlette<0.28.0,>=0.27.0->fastapi==0.95.2->-r requirements.txt (line 5)) (1.2.0)
Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->GitPython!=3.1.29,>=1.0.0->wandb->-r requirements.txt (line 20))
  Downloading smmap-5.0.1-py3-none-any.whl (24 kB)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair>=4.2.0->gradio==3.33.1->-r requirements.txt (line 7)) (2023.12.1)
Requirement already satisfied: referencing>=0.28.4 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair>=4.2.0->gradio==3.33.1->-r requirements.txt (line 7)) (0.33.0)
Requirement already satisfied: rpds-py>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair>=4.2.0->gradio==3.33.1->-r requirements.txt (line 7)) (0.17.1)
Requirement already satisfied: uc-micro-py in /usr/local/lib/python3.10/dist-packages (from linkify-it-py<3,>=1->markdown-it-py[linkify]>=2.0.0->gradio==3.33.1->-r requirements.txt (line 7)) (1.0.2)
Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard->-r requirements.txt (line 17)) (0.5.1)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<2,>=0.5->tensorboard->-r requirements.txt (line 17)) (3.2.2)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.10.0->accelerate==0.21.0->-r requirements.txt (line 1)) (1.3.0)
Building wheels for collected packages: llama-cpp-python, peft, ffmpy
  Building wheel for llama-cpp-python (pyproject.toml) ... done
  Created wheel for llama-cpp-python: filename=llama_cpp_python-0.1.77-cp310-cp310-linux_x86_64.whl size=276102 sha256=d4a18f25b6865bde63e3fadcc257c17f27f16301b63fba4f43071e3b981074e5
  Stored in directory: /root/.cache/pip/wheels/aa/ed/39/87f2ad350dbbf13b600ac744899186b8647c5323c62e2bb348
  Building wheel for peft (pyproject.toml) ... done
  Created wheel for peft: filename=peft-0.5.0.dev0-py3-none-any.whl size=73123 sha256=9293bbbd4cea4b825fdad6dc5838818a3a3dde4fa31fd96f1e90ab8013f01f74
  Stored in directory: /root/.cache/pip/wheels/ff/57/c1/a023c490307cd8ffa3b61c86c48d9767f0bb850053af18674b
  Building wheel for ffmpy (setup.py) ... done
  Created wheel for ffmpy: filename=ffmpy-0.3.1-py3-none-any.whl size=5579 sha256=d29996ef42844f560f617ef83521d97f77515d71424168d7537adaa35754fa33
  Stored in directory: /root/.cache/pip/wheels/01/a6/d1/1c0828c304a4283b2c1639a09ad86f83d7c487ef34c6b4a1bf
Successfully built llama-cpp-python peft ffmpy
Installing collected packages: tokenizers, safetensors, pydub, ffmpy, bitsandbytes, websockets, smmap, setproctitle, sentry-sdk, semantic-version, rouge, python-multipart, Pillow, orjson, markdown-it-py, llama-cpp-python-cuda, llama-cpp-python, h11, einops, docker-pycreds, dill, colorama, aiofiles, uvicorn, starlette, multiprocess, mdit-py-plugins, httpcore, gitdb, transformers, httpx, GitPython, fastapi, exllama, accelerate, wandb, peft, gradio_client, datasets, gradio, auto-gptq
  Attempting uninstall: tokenizers
    Found existing installation: tokenizers 0.15.1
    Uninstalling tokenizers-0.15.1:
      Successfully uninstalled tokenizers-0.15.1
  Attempting uninstall: safetensors
    Found existing installation: safetensors 0.4.2
    Uninstalling safetensors-0.4.2:
      Successfully uninstalled safetensors-0.4.2
  Attempting uninstall: Pillow
    Found existing installation: Pillow 9.4.0
    Uninstalling Pillow-9.4.0:
      Successfully uninstalled Pillow-9.4.0
  Attempting uninstall: markdown-it-py
    Found existing installation: markdown-it-py 3.0.0
    Uninstalling markdown-it-py-3.0.0:
      Successfully uninstalled markdown-it-py-3.0.0
  Attempting uninstall: mdit-py-plugins
    Found existing installation: mdit-py-plugins 0.4.0
    Uninstalling mdit-py-plugins-0.4.0:
      Successfully uninstalled mdit-py-plugins-0.4.0
  Attempting uninstall: transformers
    Found existing installation: transformers 4.35.2
    Uninstalling transformers-4.35.2:
      Successfully uninstalled transformers-4.35.2
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
lida 0.0.10 requires kaleido, which is not installed.
imageio 2.31.6 requires pillow<10.1.0,>=8.3.2, but you have pillow 10.2.0 which is incompatible.
Successfully installed GitPython-3.1.41 Pillow-10.2.0 accelerate-0.21.0 aiofiles-23.2.1 auto-gptq-0.3.0+cu117 bitsandbytes-0.41.0 colorama-0.4.6 datasets-2.16.1 dill-0.3.7 docker-pycreds-0.4.0 einops-0.7.0 exllama-0.0.9+cu117 fastapi-0.95.2 ffmpy-0.3.1 gitdb-4.0.11 gradio-3.33.1 gradio_client-0.2.5 h11-0.14.0 httpcore-1.0.2 httpx-0.26.0 llama-cpp-python-0.1.77 llama-cpp-python-cuda-0.1.77+cu117 markdown-it-py-2.2.0 mdit-py-plugins-0.3.3 multiprocess-0.70.15 orjson-3.9.12 peft-0.5.0.dev0 pydub-0.25.1 python-multipart-0.0.6 rouge-1.0.1 safetensors-0.3.1 semantic-version-2.10.0 sentry-sdk-1.40.0 setproctitle-1.3.3 smmap-5.0.1 starlette-0.27.0 tokenizers-0.13.3 transformers-4.31.0 uvicorn-0.27.0.post1 wandb-0.16.2 websockets-12.0
WARNING: The following packages were previously imported in this runtime:
  [PIL]
You must restart the runtime in order to use newly installed versions.
Collecting gradio==3.32.0
  Downloading gradio-3.32.0-py3-none-any.whl (19.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.9/19.9 MB 16.1 MB/s eta 0:00:00
Requirement already satisfied: aiofiles in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (23.2.1)
Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (3.9.3)
Requirement already satisfied: altair>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (4.2.2)
Requirement already satisfied: fastapi in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (0.95.2)
Requirement already satisfied: ffmpy in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (0.3.1)
Requirement already satisfied: gradio-client>=0.2.4 in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (0.2.5)
Requirement already satisfied: httpx in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (0.26.0)
Requirement already satisfied: huggingface-hub>=0.13.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (0.20.3)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (3.1.3)
Requirement already satisfied: markdown-it-py[linkify]>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (2.2.0)
Requirement already satisfied: markupsafe in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (2.1.4)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (3.7.1)
Requirement already satisfied: mdit-py-plugins<=0.3.3 in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (0.3.3)
Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (1.23.5)
Requirement already satisfied: orjson in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (3.9.12)
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (1.5.3)
Requirement already satisfied: pillow in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (10.2.0)
Requirement already satisfied: pydantic in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (1.10.14)
Requirement already satisfied: pydub in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (0.25.1)
Requirement already satisfied: pygments>=2.12.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (2.16.1)
Requirement already satisfied: python-multipart in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (0.0.6)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (6.0.1)
Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (2.31.0)
Requirement already satisfied: semantic-version in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (2.10.0)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (4.5.0)
Requirement already satisfied: uvicorn>=0.14.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (0.27.0.post1)
Requirement already satisfied: websockets>=10.0 in /usr/local/lib/python3.10/dist-packages (from gradio==3.32.0) (12.0)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio==3.32.0) (0.4)
Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio==3.32.0) (4.19.2)
Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio==3.32.0) (0.12.1)
Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from gradio-client>=0.2.4->gradio==3.32.0) (2023.6.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from gradio-client>=0.2.4->gradio==3.32.0) (23.2)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.13.0->gradio==3.32.0) (3.13.1)
Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub>=0.13.0->gradio==3.32.0) (4.66.1)
Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio==3.32.0) (0.1.2)
Requirement already satisfied: linkify-it-py<3,>=1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio==3.32.0) (2.0.2)
Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->gradio==3.32.0) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->gradio==3.32.0) (2023.4)
Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.10/dist-packages (from uvicorn>=0.14.0->gradio==3.32.0) (8.1.7)
Requirement already satisfied: h11>=0.8 in /usr/local/lib/python3.10/dist-packages (from uvicorn>=0.14.0->gradio==3.32.0) (0.14.0)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.32.0) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.32.0) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.32.0) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.32.0) (6.0.4)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.32.0) (1.9.4)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio==3.32.0) (4.0.3)
Requirement already satisfied: starlette<0.28.0,>=0.27.0 in /usr/local/lib/python3.10/dist-packages (from fastapi->gradio==3.32.0) (0.27.0)
Requirement already satisfied: anyio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio==3.32.0) (3.7.1)
Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx->gradio==3.32.0) (2023.11.17)
Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx->gradio==3.32.0) (1.0.2)
Requirement already satisfied: idna in /usr/local/lib/python3.10/dist-packages (from httpx->gradio==3.32.0) (3.6)
Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio==3.32.0) (1.3.0)
Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.32.0) (1.2.0)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.32.0) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.32.0) (4.47.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.32.0) (1.4.5)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio==3.32.0) (3.1.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->gradio==3.32.0) (3.3.2)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->gradio==3.32.0) (2.0.7)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair>=4.2.0->gradio==3.32.0) (2023.12.1)
Requirement already satisfied: referencing>=0.28.4 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair>=4.2.0->gradio==3.32.0) (0.33.0)
Requirement already satisfied: rpds-py>=0.7.1 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair>=4.2.0->gradio==3.32.0) (0.17.1)
Requirement already satisfied: uc-micro-py in /usr/local/lib/python3.10/dist-packages (from linkify-it-py<3,>=1->markdown-it-py[linkify]>=2.0.0->gradio==3.32.0) (1.0.2)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.1->pandas->gradio==3.32.0) (1.16.0)
Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio->httpx->gradio==3.32.0) (1.2.0)
Installing collected packages: gradio
  Attempting uninstall: gradio
    Found existing installation: gradio 3.33.1
    Uninstalling gradio-3.33.1:
      Successfully uninstalled gradio-3.33.1
Successfully installed gradio-3.32.0
Found existing installation: llama_cpp_python 0.1.77
Uninstalling llama_cpp_python-0.1.77:
  Successfully uninstalled llama_cpp_python-0.1.77
Collecting llama-cpp-python==0.1.78
  Downloading llama_cpp_python-0.1.78.tar.gz (1.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 9.9 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: typing-extensions>=4.5.0 in /usr/local/lib/python3.10/dist-packages (from llama-cpp-python==0.1.78) (4.5.0)
Requirement already satisfied: numpy>=1.20.0 in /usr/local/lib/python3.10/dist-packages (from llama-cpp-python==0.1.78) (1.23.5)
Requirement already satisfied: diskcache>=5.6.1 in /usr/local/lib/python3.10/dist-packages (from llama-cpp-python==0.1.78) (5.6.3)
Building wheels for collected packages: llama-cpp-python
  Building wheel for llama-cpp-python (pyproject.toml) ... done
  Created wheel for llama-cpp-python: filename=llama_cpp_python-0.1.78-cp310-cp310-linux_x86_64.whl size=5811091 sha256=209d670a27a6691b40380caee617af17d99b896bfef56b64fabfce5f603fd2db
  Stored in directory: /tmp/pip-ephem-wheel-cache-rhs2coaq/wheels/61/f9/20/9ca660a9d3f2a47e44217059409478865948b5c8a1cba70030
Successfully built llama-cpp-python
Installing collected packages: llama-cpp-python
Successfully installed llama-cpp-python-0.1.78

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
f1cf97|OK  |   164MiB/s|/content/text-generation-webui/models//wizardlm-1.0-uncensored-llama2-13b.ggmlv3.q5_K_M.bin

Status Legend:
(OK):download completed.
/content/text-generation-webui
2024-02-03 08:22:25 WARNING:The gradio "share link" feature uses a proprietary executable to create a reverse tunnel. Use it with care.
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cextension.py:34: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.
  warn("The installed version of bitsandbytes was compiled without GPU support. "
/usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cpu.so: undefined symbol: cadam32bit_grad_fp32
2024-02-03 08:22:29.859509: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-02-03 08:22:29.859583: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-02-03 08:22:29.861007: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-02-03 08:22:31.839727: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
2024-02-03 08:22:33 INFO:Loading wizardlm-1.0-uncensored-llama2-13b.ggmlv3.q5_K_M.bin...
CUDA error 35 at /tmp/pip-install-nydaa5se/llama-cpp-python_720d0f2868d94d788fd2ec34324b390a/vendor/llama.cpp/ggml-cuda.cu:4883: CUDA driver version is insufficient for CUDA runtime version
Troyanovsky commented 9 months ago

Fixed by updating webui and changing model to GGUF format.