didn't work 😢

[Filarh]

import os

user_name = "anon8231489123" #@param {"type": "string"}

model_name = "gpt4-x-alpaca-13b-native-4bit-128g" #@param {"type": "string"}

!apt-get -y install -qq aria2
!git clone -b v1.0 https://github.com/camenduru/text-generation-webui
%cd /content/text-generation-webui
!pip install -r requirements.txt

models_path = "/content/text-generation-webui/models/"
model_path = os.path.join(models_path, model_name)
os.makedirs(model_path, exist_ok=True)

!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/{user_name}/{model_name}/raw/main/config.json -d {model_path} -o config.json
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/{user_name}/{model_name}/raw/main/generation_config.json -d {model_path} -o generation_config.json
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/{user_name}/{model_name}/raw/main/pytorch_model.bin.index.json -d {model_path} -o pytorch_model.bin.index.json
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/{user_name}/{model_name}/raw/main/special_tokens_map.json -d {model_path} -o special_tokens_map.json
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/{user_name}/{model_name}/resolve/main/tokenizer.model -d {model_path} -o tokenizer.model
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/{user_name}/{model_name}/raw/main/tokenizer_config.json -d {model_path} -o tokenizer_config.json
!aria2c --console-log-level=error -c -x 16 -s 16 -k 1M https://huggingface.co/{user_name}/{model_name}/resolve/main/gpt-x-alpaca-13b-native-4bit-128g-cuda.pt -d {model_path} -o {model_name}.pt
%cd /content/text-generation-webui
!python server.py --share --chat --wbits 4 --groupsize 128 --model {model_name}

got:

Loading gpt4-x-alpaca-13b-native-4bit-128g... Loading model ... ^C

didn't try on colab pro, is there a way to optimize this?

edit: i just found this:

tsumeone/gpt4-x-alpaca-13b-native-4bit-128g-cuda

[camenduru]

thanks ❤ I will switch the model