Running SliceGPT experiment.
PyTorch device: cuda:0
Number of available cuda devices: 1
Loading /home/iotsc01/llama-7b-hf config and model weights from /home/iotsc01/xinpengq/LLM-Pruner-main/llama-7b-hf
Traceback (most recent call last):
File "/home/iotsc01/TransformerCompression-main/experiments/run_slicegpt.py", line 274, in <module>
slicing_main(slicing_args)
File "/home/iotsc01/TransformerCompression-main/experiments/run_slicegpt.py", line 149, in slicing_main
model_adapter, tokenizer = hf_utils.get_model_and_tokenizer(
File "/home/iotsc01/TransformerCompression-main/src/slicegpt/hf_utils.py", line 33, in wrapper
result = func(*args, **kwargs)
File "/home/iotsc01/TransformerCompression-main/src/slicegpt/hf_utils.py", line 91, in get_model_and_tokenizer
model_adapter = ModelAdapter.from_model(
File "/home/iotsc01/TransformerCompression-main/src/slicegpt/model_adapter.py", line 361, in from_model
raise NotImplementedError(f"{model_path} is neither a Hugging Face model nor a supported local model.")
NotImplementedError: /home/iotsc01/llama-7b-hf is neither a Hugging Face model nor a supported local model.
@mtodd @tpope @sverrejoh @radical @ijbaird When use the following command to prune the model
llama-7b
:the following problem happend:
How can i solve it?