search

oobabooga / text-generation-webui

A Gradio web UI for Large Language Models.
GNU Affero General Public License v3.0
40.73k stars 5.32k forks source link

Loras are no longer automatically loaded on startup when specified using --lora #1550

Closed ragnarkar closed 1 year ago

ragnarkar commented 1 year ago

Describe the bug

When I launch the server and specify a Lora to load on startup, it doesn't get loaded. Previously, I would see a message in the console saying that the lora is loaded when I run using:

!python server.py --share --load-in-8bit --lora-dir loras --lora $dirname

But now, I don't see it and my model behaves as if there's no lora loaded.

Even worse, when I go into the Model settings to manually load the Lora, the app reports that the lora is loaded successfully but still behaves as if the lora isn't loaded.

Is there an existing issue for this?

Reproduction

Here is a simple Colab notebook for a project I'm currently working on. The latest version of text-generation-webui doesn't load the loras anymore on startup and cannot load any even after I force it to by going into the Model settings:

https://github.com/ragnarkar/reddit_imitator/blob/main/Reddit_imitator.ipynb

The following code doesn't work:

%cd /content
!git clone https://github.com/oobabooga/text-generation-webui
!git clone https://github.com/ragnarkar/reddit_imitator
%cd text-generation-webui
!rm prompts/*
!cp ../reddit_imitator/Prompts/* prompts/
!python download-model.py facebook/opt-2.7b
!pip install -r requirements.txt 
subreddit = "chatgpt"
!python download-model.py $full_path --output loras
dirname = full_path.replace("/","_")
!python server.py --share --load-in-8bit --lora-dir loras --lora $dirname

But the following code works fine because I ask it to git checkout a previous commit that was working:

%cd /content
!git clone https://github.com/oobabooga/text-generation-webui
!git clone https://github.com/ragnarkar/reddit_imitator
%cd text-generation-webui
!git checkout c86e9a3372c078bfae9bee8dfc147583b3f30b8c
!rm prompts/*
!cp ../reddit_imitator/Prompts/* prompts/
!python download-model.py facebook/opt-2.7b
!pip install -r requirements.txt 
subreddit = "chatgpt"
!python download-model.py $full_path --output loras
dirname = full_path.replace("/","_")
!python server.py --share --load-in-8bit --lora-dir loras --lora $dirname

When you run the 2nd code block in Colab, notice an extra line in the output shortly before your Gradio link:

Applying the following LoRAs to facebook_opt-2.7b: ragnarkar_subreddit_chatgpt_opt-2.7b_lora

Screenshot

No response

Logs

/content
Cloning into 'text-generation-webui'...
remote: Enumerating objects: 6234, done.
remote: Counting objects: 100% (1309/1309), done.
remote: Compressing objects: 100% (275/275), done.
remote: Total 6234 (delta 1136), reused 1150 (delta 1030), pack-reused 4925
Receiving objects: 100% (6234/6234), 2.31 MiB | 23.18 MiB/s, done.
Resolving deltas: 100% (4144/4144), done.
Cloning into 'reddit_imitator'...
remote: Enumerating objects: 56, done.
remote: Counting objects: 100% (56/56), done.
remote: Compressing objects: 100% (54/54), done.
remote: Total 56 (delta 30), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (56/56), 28.25 KiB | 556.00 KiB/s, done.
/content/text-generation-webui
Downloading the model to models/facebook_opt-2.7b
100% 8.78k/8.78k [00:00<00:00, 6.53MiB/s]
100% 691/691 [00:00<00:00, 550kiB/s]
100% 137/137 [00:00<00:00, 109kiB/s]
100% 456k/456k [00:00<00:00, 20.8MiB/s]
100% 5.30G/5.30G [01:09<00:00, 76.2MiB/s]
100% 441/441 [00:00<00:00, 350kiB/s]
100% 685/685 [00:00<00:00, 558kiB/s]
100% 899k/899k [00:00<00:00, 1.07MiB/s]
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting git+https://github.com/huggingface/peft (from -r requirements.txt (line 16))
  Cloning https://github.com/huggingface/peft to /tmp/pip-req-build-e46o4iov
  Running command git clone --filter=blob:none --quiet https://github.com/huggingface/peft /tmp/pip-req-build-e46o4iov
  Resolved https://github.com/huggingface/peft to commit 49a20c16dcd9de5716feee717e8eb742efb9eff9
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Ignoring llama-cpp-python: markers 'platform_system == "Windows"' don't match your environment
Collecting accelerate==0.18.0
  Downloading accelerate-0.18.0-py3-none-any.whl (215 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 215.3/215.3 kB 18.1 MB/s eta 0:00:00
Collecting colorama
  Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)
Collecting datasets
  Downloading datasets-2.11.0-py3-none-any.whl (468 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 468.7/468.7 kB 43.6 MB/s eta 0:00:00
Collecting flexgen==0.1.7
  Downloading flexgen-0.1.7-py3-none-any.whl (50 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.8/50.8 kB 7.0 MB/s eta 0:00:00
Collecting gradio==3.25.0
  Downloading gradio-3.25.0-py3-none-any.whl (17.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.5/17.5 MB 80.1 MB/s eta 0:00:00
Requirement already satisfied: markdown in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 6)) (3.4.3)
Requirement already satisfied: numpy in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 7)) (1.22.4)
Requirement already satisfied: pandas in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 8)) (1.5.3)
Collecting Pillow>=9.5.0
  Downloading Pillow-9.5.0-cp39-cp39-manylinux_2_28_x86_64.whl (3.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 103.1 MB/s eta 0:00:00
Requirement already satisfied: pyyaml in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 10)) (6.0)
Requirement already satisfied: requests in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 11)) (2.27.1)
Collecting rwkv==0.7.3
  Downloading rwkv-0.7.3-py3-none-any.whl (16 kB)
Collecting safetensors==0.3.0
  Downloading safetensors-0.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 31.0 MB/s eta 0:00:00
Collecting sentencepiece
  Downloading sentencepiece-0.1.98-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 80.4 MB/s eta 0:00:00
Requirement already satisfied: tqdm in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 15)) (4.65.0)
Collecting transformers==4.28.1
  Downloading transformers-4.28.1-py3-none-any.whl (7.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.0/7.0 MB 106.5 MB/s eta 0:00:00
Collecting bitsandbytes==0.38.1
  Downloading bitsandbytes-0.38.1-py3-none-any.whl (104.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 104.3/104.3 MB 10.6 MB/s eta 0:00:00
Collecting llama-cpp-python==0.1.36
  Downloading llama_cpp_python-0.1.36.tar.gz (1.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 73.4 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.9/dist-packages (from accelerate==0.18.0->-r requirements.txt (line 1)) (23.1)
Requirement already satisfied: torch>=1.4.0 in /usr/local/lib/python3.9/dist-packages (from accelerate==0.18.0->-r requirements.txt (line 1)) (2.0.0+cu118)
Requirement already satisfied: psutil in /usr/local/lib/python3.9/dist-packages (from accelerate==0.18.0->-r requirements.txt (line 1)) (5.9.5)
Collecting pulp
  Downloading PuLP-2.7.0-py3-none-any.whl (14.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.3/14.3 MB 34.6 MB/s eta 0:00:00
Requirement already satisfied: attrs in /usr/local/lib/python3.9/dist-packages (from flexgen==0.1.7->-r requirements.txt (line 4)) (23.1.0)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.9/dist-packages (from gradio==3.25.0->-r requirements.txt (line 5)) (3.7.1)
Collecting huggingface-hub>=0.13.0
  Downloading huggingface_hub-0.14.1-py3-none-any.whl (224 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 224.5/224.5 kB 28.7 MB/s eta 0:00:00
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.9/dist-packages (from gradio==3.25.0->-r requirements.txt (line 5)) (4.5.0)
Collecting ffmpy
  Downloading ffmpy-0.3.0.tar.gz (4.8 kB)
  Preparing metadata (setup.py) ... done
Collecting httpx
  Downloading httpx-0.24.0-py3-none-any.whl (75 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 75.3/75.3 kB 10.8 MB/s eta 0:00:00
Collecting aiofiles
  Downloading aiofiles-23.1.0-py3-none-any.whl (14 kB)
Collecting pydub
  Downloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)
Collecting gradio-client>=0.0.8
  Downloading gradio_client-0.1.3-py3-none-any.whl (286 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 286.2/286.2 kB 31.0 MB/s eta 0:00:00
Collecting fastapi
  Downloading fastapi-0.95.1-py3-none-any.whl (56 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.0/57.0 kB 7.5 MB/s eta 0:00:00
Collecting python-multipart
  Downloading python_multipart-0.0.6-py3-none-any.whl (45 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.7/45.7 kB 5.9 MB/s eta 0:00:00
Collecting uvicorn
  Downloading uvicorn-0.21.1-py3-none-any.whl (57 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.8/57.8 kB 6.5 MB/s eta 0:00:00
Requirement already satisfied: altair>=4.2.0 in /usr/local/lib/python3.9/dist-packages (from gradio==3.25.0->-r requirements.txt (line 5)) (4.2.2)
Requirement already satisfied: markupsafe in /usr/local/lib/python3.9/dist-packages (from gradio==3.25.0->-r requirements.txt (line 5)) (2.1.2)
Collecting orjson
  Downloading orjson-3.8.10-cp39-cp39-manylinux_2_28_x86_64.whl (140 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.5/140.5 kB 16.8 MB/s eta 0:00:00
Requirement already satisfied: jinja2 in /usr/local/lib/python3.9/dist-packages (from gradio==3.25.0->-r requirements.txt (line 5)) (3.1.2)
Requirement already satisfied: markdown-it-py[linkify]>=2.0.0 in /usr/local/lib/python3.9/dist-packages (from gradio==3.25.0->-r requirements.txt (line 5)) (2.2.0)
Collecting semantic-version
  Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)
Collecting websockets>=10.0
  Downloading websockets-11.0.2-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (129 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 129.7/129.7 kB 16.8 MB/s eta 0:00:00
Collecting aiohttp
  Downloading aiohttp-3.8.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.0/1.0 MB 10.1 MB/s eta 0:00:00
Collecting mdit-py-plugins<=0.3.3
  Downloading mdit_py_plugins-0.3.3-py3-none-any.whl (50 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.5/50.5 kB 6.9 MB/s eta 0:00:00
Requirement already satisfied: pydantic in /usr/local/lib/python3.9/dist-packages (from gradio==3.25.0->-r requirements.txt (line 5)) (1.10.7)
Collecting tokenizers>=0.13.2
  Downloading tokenizers-0.13.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.8/7.8 MB 70.0 MB/s eta 0:00:00
Requirement already satisfied: filelock in /usr/local/lib/python3.9/dist-packages (from transformers==4.28.1->-r requirements.txt (line 17)) (3.11.0)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.9/dist-packages (from transformers==4.28.1->-r requirements.txt (line 17)) (2022.10.31)
Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.9/dist-packages (from datasets->-r requirements.txt (line 3)) (9.0.0)
Requirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.9/dist-packages (from datasets->-r requirements.txt (line 3)) (2023.4.0)
Collecting multiprocess
  Downloading multiprocess-0.70.14-py39-none-any.whl (132 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.9/132.9 kB 16.2 MB/s eta 0:00:00
Collecting responses<0.19
  Downloading responses-0.18.0-py3-none-any.whl (38 kB)
Collecting xxhash
  Downloading xxhash-3.2.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (212 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 212.2/212.2 kB 28.5 MB/s eta 0:00:00
Collecting dill<0.3.7,>=0.3.0
  Downloading dill-0.3.6-py3-none-any.whl (110 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 110.5/110.5 kB 15.2 MB/s eta 0:00:00
Requirement already satisfied: importlib-metadata>=4.4 in /usr/local/lib/python3.9/dist-packages (from markdown->-r requirements.txt (line 6)) (6.4.1)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.9/dist-packages (from pandas->-r requirements.txt (line 8)) (2022.7.1)
Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.9/dist-packages (from pandas->-r requirements.txt (line 8)) (2.8.2)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests->-r requirements.txt (line 11)) (1.26.15)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests->-r requirements.txt (line 11)) (3.4)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.9/dist-packages (from requests->-r requirements.txt (line 11)) (2.0.12)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests->-r requirements.txt (line 11)) (2022.12.7)
Requirement already satisfied: toolz in /usr/local/lib/python3.9/dist-packages (from altair>=4.2.0->gradio==3.25.0->-r requirements.txt (line 5)) (0.12.0)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.9/dist-packages (from altair>=4.2.0->gradio==3.25.0->-r requirements.txt (line 5)) (0.4)
Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.9/dist-packages (from altair>=4.2.0->gradio==3.25.0->-r requirements.txt (line 5)) (4.3.3)
Collecting aiosignal>=1.1.2
  Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Collecting frozenlist>=1.1.1
  Downloading frozenlist-1.3.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (158 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 158.8/158.8 kB 22.5 MB/s eta 0:00:00
Collecting yarl<2.0,>=1.0
  Downloading yarl-1.9.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (269 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 269.4/269.4 kB 34.2 MB/s eta 0:00:00
Collecting async-timeout<5.0,>=4.0.0a3
  Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting multidict<7.0,>=4.5
  Downloading multidict-6.0.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 kB 15.0 MB/s eta 0:00:00
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.9/dist-packages (from importlib-metadata>=4.4->markdown->-r requirements.txt (line 6)) (3.15.0)
Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.9/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio==3.25.0->-r requirements.txt (line 5)) (0.1.2)
Collecting linkify-it-py<3,>=1
  Downloading linkify_it_py-2.0.0-py3-none-any.whl (19 kB)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.9/dist-packages (from python-dateutil>=2.8.1->pandas->-r requirements.txt (line 8)) (1.16.0)
Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.9/dist-packages (from torch>=1.4.0->accelerate==0.18.0->-r requirements.txt (line 1)) (2.0.0)
Requirement already satisfied: networkx in /usr/local/lib/python3.9/dist-packages (from torch>=1.4.0->accelerate==0.18.0->-r requirements.txt (line 1)) (3.1)
Requirement already satisfied: sympy in /usr/local/lib/python3.9/dist-packages (from torch>=1.4.0->accelerate==0.18.0->-r requirements.txt (line 1)) (1.11.1)
Requirement already satisfied: lit in /usr/local/lib/python3.9/dist-packages (from triton==2.0.0->torch>=1.4.0->accelerate==0.18.0->-r requirements.txt (line 1)) (16.0.1)
Requirement already satisfied: cmake in /usr/local/lib/python3.9/dist-packages (from triton==2.0.0->torch>=1.4.0->accelerate==0.18.0->-r requirements.txt (line 1)) (3.25.2)
Collecting starlette<0.27.0,>=0.26.1
  Downloading starlette-0.26.1-py3-none-any.whl (66 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.9/66.9 kB 10.4 MB/s eta 0:00:00
Requirement already satisfied: sniffio in /usr/local/lib/python3.9/dist-packages (from httpx->gradio==3.25.0->-r requirements.txt (line 5)) (1.3.0)
Collecting httpcore<0.18.0,>=0.15.0
  Downloading httpcore-0.17.0-py3-none-any.whl (70 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 70.6/70.6 kB 7.0 MB/s eta 0:00:00
Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.9/dist-packages (from matplotlib->gradio==3.25.0->-r requirements.txt (line 5)) (1.0.7)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.9/dist-packages (from matplotlib->gradio==3.25.0->-r requirements.txt (line 5)) (4.39.3)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.9/dist-packages (from matplotlib->gradio==3.25.0->-r requirements.txt (line 5)) (0.11.0)
Requirement already satisfied: importlib-resources>=3.2.0 in /usr/local/lib/python3.9/dist-packages (from matplotlib->gradio==3.25.0->-r requirements.txt (line 5)) (5.12.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.9/dist-packages (from matplotlib->gradio==3.25.0->-r requirements.txt (line 5)) (1.4.4)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.9/dist-packages (from matplotlib->gradio==3.25.0->-r requirements.txt (line 5)) (3.0.9)
Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.9/dist-packages (from uvicorn->gradio==3.25.0->-r requirements.txt (line 5)) (8.1.3)
Collecting h11>=0.8
  Downloading h11-0.14.0-py3-none-any.whl (58 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 8.1 MB/s eta 0:00:00
Requirement already satisfied: anyio<5.0,>=3.0 in /usr/local/lib/python3.9/dist-packages (from httpcore<0.18.0,>=0.15.0->httpx->gradio==3.25.0->-r requirements.txt (line 5)) (3.6.2)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.9/dist-packages (from jsonschema>=3.0->altair>=4.2.0->gradio==3.25.0->-r requirements.txt (line 5)) (0.19.3)
Collecting uc-micro-py
  Downloading uc_micro_py-1.0.1-py3-none-any.whl (6.2 kB)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.9/dist-packages (from sympy->torch>=1.4.0->accelerate==0.18.0->-r requirements.txt (line 1)) (1.3.0)
Building wheels for collected packages: llama-cpp-python, peft, ffmpy
  Building wheel for llama-cpp-python (pyproject.toml) ... done
  Created wheel for llama-cpp-python: filename=llama_cpp_python-0.1.36-cp39-cp39-linux_x86_64.whl size=161748 sha256=0d7eb7d387fb5cb3df13c2188e0d00ac137bee7f3f0a4df700ba6dc14a9e9c95
  Stored in directory: /root/.cache/pip/wheels/81/ce/36/e1ac06d0b5e179b73d148bb80b00363d88831e9a6504ebfcb5
  Building wheel for peft (pyproject.toml) ... done
  Created wheel for peft: filename=peft-0.3.0.dev0-py3-none-any.whl size=55512 sha256=7e9e17951bfd64ef822cddbd3f83ef14107518e347a1199082b96be8d5eae22e
  Stored in directory: /tmp/pip-ephem-wheel-cache-y5rr482o/wheels/45/06/33/0048c03714539b315d99beef9ea6b6dad0fd5750105e221583
  Building wheel for ffmpy (setup.py) ... done
  Created wheel for ffmpy: filename=ffmpy-0.3.0-py3-none-any.whl size=4707 sha256=1fbdd6509fed545d2f848bccb49bf695fd1da32c98d0733b830c0c8c608e808e
  Stored in directory: /root/.cache/pip/wheels/91/e2/96/f676aa08bfd789328c6576cd0f1fde4a3d686703bb0c247697
Successfully built llama-cpp-python peft ffmpy
Installing collected packages: tokenizers, sentencepiece, safetensors, pydub, pulp, ffmpy, bitsandbytes, xxhash, websockets, uc-micro-py, semantic-version, rwkv, python-multipart, Pillow, orjson, multidict, llama-cpp-python, h11, frozenlist, dill, colorama, async-timeout, aiofiles, yarl, uvicorn, starlette, responses, multiprocess, mdit-py-plugins, linkify-it-py, huggingface-hub, httpcore, aiosignal, transformers, httpx, fastapi, aiohttp, gradio-client, gradio, datasets, accelerate, peft, flexgen
  Attempting uninstall: Pillow
    Found existing installation: Pillow 8.4.0
    Uninstalling Pillow-8.4.0:
      Successfully uninstalled Pillow-8.4.0
Successfully installed Pillow-9.5.0 accelerate-0.18.0 aiofiles-23.1.0 aiohttp-3.8.4 aiosignal-1.3.1 async-timeout-4.0.2 bitsandbytes-0.38.1 colorama-0.4.6 datasets-2.11.0 dill-0.3.6 fastapi-0.95.1 ffmpy-0.3.0 flexgen-0.1.7 frozenlist-1.3.3 gradio-3.25.0 gradio-client-0.1.3 h11-0.14.0 httpcore-0.17.0 httpx-0.24.0 huggingface-hub-0.14.1 linkify-it-py-2.0.0 llama-cpp-python-0.1.36 mdit-py-plugins-0.3.3 multidict-6.0.4 multiprocess-0.70.14 orjson-3.8.10 peft-0.3.0.dev0 pulp-2.7.0 pydub-0.25.1 python-multipart-0.0.6 responses-0.18.0 rwkv-0.7.3 safetensors-0.3.0 semantic-version-2.10.0 sentencepiece-0.1.98 starlette-0.26.1 tokenizers-0.13.3 transformers-4.28.1 uc-micro-py-1.0.1 uvicorn-0.21.1 websockets-11.0.2 xxhash-3.2.0 yarl-1.9.2
WARNING: The following packages were previously imported in this runtime:
  [PIL]
You must restart the runtime in order to use newly installed versions.

Downloading the model to loras/ragnarkar_subreddit_chatgpt_opt-2.7b_lora
100% 30.0/30.0 [00:00<00:00, 24.4kiB/s]
100% 338/338 [00:00<00:00, 155kiB/s]
100% 5.29M/5.29M [00:01<00:00, 3.56MiB/s]

Gradio HTTP request redirected to localhost :)
Warning: the gradio "share link" feature downloads a proprietary and
unaudited blob to create a reverse tunnel. This is potentially dangerous.

2023-04-26 01:12:40.850935: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
bin /usr/local/lib/python3.9/dist-packages/bitsandbytes/libbitsandbytes_cuda118.so
Loading facebook_opt-2.7b...
Loaded the model in 49.42 seconds.
Running on local URL:  http://127.0.0.1:7860
Running on public URL: https://42b95f4919825361cc.gradio.live

This share link expires in 72 hours. For free permanent hosting and GPU upgrades (NEW!), check out Spaces: https://huggingface.co/spaces

System Info

FREE Google Colab, 15GB VRAM (T4), 12.7GB RAM
oobabooga commented 1 year ago

Fixed here https://github.com/oobabooga/text-generation-webui/commit/f39c99fa14901f649ef9fffa00caa80155921d3e