BatsResearch / bonito

A lightweight library for generating synthetic instruction tuning datasets for your data without GPT.
BSD 3-Clause "New" or "Revised" License
702 stars 46 forks source link

An error occurred when running 'bonito = QuantizedBonito("alexandreteles/bonito-v1-awq")' #27

Closed EASTghr closed 4 months ago

EASTghr commented 4 months ago

I encountered this error while following the Quantized Bonito Tutorial and I would like to know what caused this error

/opt/anaconda3/envs/bonito/bin/python /home/easyai/Downloads/BONITO/bonito/1.py /opt/anaconda3/envs/bonito/lib/python3.9/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: resume_download is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use force_download=True. warnings.warn( Replacing layers...: 100%|██████████| 32/32 [00:03<00:00, 8.10it/s] Traceback (most recent call last): File "/home/easyai/Downloads/BONITO/bonito/1.py", line 95, in bonito = QuantizedBonito("alexandreteles/bonito-v1-awq") File "/home/easyai/Downloads/BONITO/bonito/1.py", line 10, in init self.model = AutoAWQForCausalLM.from_quantized( File "/opt/anaconda3/envs/bonito/lib/python3.9/site-packages/awq/models/auto.py", line 101, in from_quantized return AWQ_CAUSAL_LM_MODEL_MAP[model_type].from_quantized( File "/opt/anaconda3/envs/bonito/lib/python3.9/site-packages/awq/models/base.py", line 454, in from_quantized load_checkpoint_and_dispatch( File "/opt/anaconda3/envs/bonito/lib/python3.9/site-packages/accelerate/big_modeling.py", line 607, in load_checkpoint_and_dispatch load_checkpoint_in_model( File "/opt/anaconda3/envs/bonito/lib/python3.9/site-packages/accelerate/utils/modeling.py", line 1699, in load_checkpoint_in_model raise ValueError( ValueError: /home/easyai/.cache/huggingface/hub/models--alexandreteles--bonito-v1-awq/snapshots/d0b7ae8fd6606d9f78038304dc75edb14d6a3757 is not a folder containing a .index.json file or a pytorch_model.bin or a model.safetensors file

Process finished with exit code 1