Closed sofq closed 1 year ago
I got this error when try to run server.py on Colab. Did I set up something wrong?
Monkey Patch Completed. Loading ../llama-13b-4bit.pt ... Loading Model ... ╭───────────────────── Traceback (most recent call last) ──────────────────────╮ │ /content/alpaca_lora_4bit/text-generation-webui/server.py:236 in <module> │ │ │ │ 233 │ │ i = int(input())-1 │ │ 234 │ │ print() │ │ 235 │ shared.model_name = available_models[i] │ │ ❱ 236 shared.model, shared.tokenizer = load_model(shared.model_name) │ │ 237 if shared.args.lora: │ │ 238 │ add_lora_to_model(shared.args.lora) │ │ 239 │ │ │ │ /content/alpaca_lora_4bit/text-generation-webui/custom_monkey_patch.py:21 in │ │ load_model_llama │ │ │ │ 18 │ print("Loading {} ...".format(model_path)) │ │ 19 │ t0 = time.time() │ │ 20 │ │ │ ❱ 21 │ model, tokenizer = load_llama_model_4bit_low_ram(config_path, model │ │ 22 │ │ │ 23 │ model = PeftModel.from_pretrained(model, lora_path, device_map={'': │ │ 24 │ print('{} Lora Applied.'.format(lora_path)) │ │ │ │ /content/alpaca_lora_4bit/text-generation-webui/../repository/GPTQ-for-LLaMa │ │ /autograd_4bit.py:222 in load_llama_model_4bit_low_ram │ │ │ │ 219 │ │ │ if name in layers: │ │ 220 │ │ │ │ del layers[name] │ │ 221 │ │ make_quant_for_4bit_autograd(model, layers) │ │ ❱ 222 │ model = accelerate.load_checkpoint_and_dispatch( │ │ 223 │ │ model=model, │ │ 224 │ │ checkpoint=model_path, │ │ 225 │ │ device_map=device_map, │ │ │ │ /usr/local/lib/python3.9/dist-packages/accelerate/big_modeling.py:479 in │ │ load_checkpoint_and_dispatch │ │ │ │ 476 │ │ ) │ │ 477 │ if offload_state_dict is None and "disk" in device_map.values(): │ │ 478 │ │ offload_state_dict = True │ │ ❱ 479 │ load_checkpoint_in_model( │ │ 480 │ │ model, │ │ 481 │ │ checkpoint, │ │ 482 │ │ device_map=device_map, │ │ │ │ /usr/local/lib/python3.9/dist-packages/accelerate/utils/modeling.py:899 in │ │ load_checkpoint_in_model │ │ │ │ 896 │ │ else: │ │ 897 │ │ │ raise ValueError(f"{checkpoint} containing more than one ` │ │ 898 │ else: │ │ ❱ 899 │ │ raise ValueError( │ │ 900 │ │ │ "`checkpoint` should be the path to a file containing a wh │ │ 901 │ │ │ f"checkpoint, or a folder containing a sharded checkpoint, │ │ 902 │ │ ) │ ╰──────────────────────────────────────────────────────────────────────────────╯ ValueError: `checkpoint` should be the path to a file containing a whole state dict, or the index of a sharded checkpoint, or a folder containing a sharded checkpoint, but got ../llama-13b-4bit.pt.
My silly, I set wrong path, update model_path in custom_monkey_patch.py to model_path = 'llama-13b-4bit.pt' fixed the problem.
custom_monkey_patch.py
model_path = 'llama-13b-4bit.pt'
I got this error when try to run server.py on Colab. Did I set up something wrong?