File ~/anaconda3/envs/study1/lib/python3.9/site-packages/transformers/modeling_utils.py:1885, in PreTrainedModel.save_pretrained(self, save_directory, is_main_process, state_dict, save_function, push_to_hub, max_shard_size, safe_serialization, variant, token, save_peft_format, **kwargs)
1881 if _hf_peft_config_loaded:
1882 logger.info(
1883 "Detected adapters on the model, saving the model in the PEFT format, only adapter weights will be saved."
1884 )
-> 1885 state_dict = model_to_save.get_adapter_state_dict()
1887 if save_peft_format:
1888 logger.info(
1889 "To match the expected format of the PEFT library, all keys of the state dict of adapters will be pre-pended with base_model.model."
1890 )
MurraryZhao
commented
9 months ago
8bit量化后用save_pretrained方法保存报错 ` from transformers import AutoModelForCausalLM, AutoTokenizer import torch
model_path = 'xxx/7B-Base' save_dir = '7B-Base/8bits/'
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.float16, trust_remote_code=True) model = model.quantize(8).cuda()
tokenizer.save_pretrained(save_dir) model.save_pretrained(save_dir) `
` KeyError Traceback (most recent call last) Cell In[2], line 5 3 save_dir = '7B-Base/8bits/' 4 tokenizer.save_pretrained(save_dir) ----> 5 model.save_pretrained(save_dir)
File ~/anaconda3/envs/study1/lib/python3.9/site-packages/transformers/modeling_utils.py:1885, in PreTrainedModel.save_pretrained(self, save_directory, is_main_process, state_dict, save_function, push_to_hub, max_shard_size, safe_serialization, variant, token, save_peft_format, **kwargs) 1881 if _hf_peft_config_loaded: 1882 logger.info( 1883 "Detected adapters on the model, saving the model in the PEFT format, only adapter weights will be saved." 1884 ) -> 1885 state_dict = model_to_save.get_adapter_state_dict() 1887 if save_peft_format: 1888 logger.info( 1889 "To match the expected format of the PEFT library, all keys of the state dict of adapters will be pre-pended with
base_model.model." 1890 )File ~/anaconda3/envs/study1/lib/python3.9/site-packages/transformers/integrations/peft.py:338, in PeftAdapterMixin.get_adapter_state_dict(self, adapter_name) 335 if adapter_name is None: 336 adapter_name = self.active_adapter() --> 338 adapter_state_dict = get_peft_model_state_dict(self, adapter_name=adapter_name) 339 return adapter_state_dict
File ~/anaconda3/envs/study1/lib/python3.9/site-packages/peft/utils/save_and_load.py:39, in get_peft_model_state_dict(model, state_dict, adapter_name) ... ---> 39 config = model.peft_config[adapter_name] 40 if state_dict is None: 41 state_dict = model.state_dict()
KeyError: None `