AutoModelForCausalLM

[Altrouge7]

AutoModelForCausalLM 中class没有chatglm你是如何解决的呢

[Linjiahua]

我也有同样的问题

[yangzhipeng1108]

更新最新版的transformer 和 最新版的chatglm

[Altrouge7]

transformers已经是4.30.2版本了，我通过修改chatglm的config.json解决了这个问题，但是出现在model_utils.py中出现了另一个问题：Traceback (most recent call last): File "/home/rlhf/DeepSpeedExamples/applications/DeepSpeed-Chat/training/step1_supervised_finetuning/main.py", line 356, in main() File "/home/rlhf/DeepSpeedExamples/applications/DeepSpeed-Chat/training/step1_supervised_finetuning/main.py", line 220, in main model = create_hf_model(AutoModelForCausalLM, File "/home/rlhf/DeepSpeedExamples/applications/DeepSpeed-Chat/training/utils/model/model_utils.py", line 49, in create_hf_model model.resize_token_embeddings(int( File "/home/anaconda3/envs/rl/lib/python3.9/site-packages/transformers/modeling_utils.py", line 1395, in resize_token_embeddings model_embeds = self._resize_token_embeddings(new_num_tokens) File "/home/anaconda3/envs/rl/lib/python3.9/site-packages/transformers/modeling_utils.py", line 1411, in _resize_token_embeddings self.set_input_embeddings(new_embeddings) File "/home/anaconda3/envs/rl/lib/python3.9/site-packages/transformers/modeling_utils.py", line 1236, in set_input_embeddings base_model.set_input_embeddings(value) File "/home/anaconda3/envs/rl/lib/python3.9/site-packages/transformers/modeling_utils.py", line 1238, in set_input_embeddings raise NotImplementedError NotImplementedError 请问这是怎么解决的呢

[newtonysls]

transformers已经是4.30.2版本了，我通过修改chatglm的config.json解决了这个问题，但是出现在model_utils.py中出现了另一个问题：Traceback (most recent call last): File "/home/rlhf/DeepSpeedExamples/applications/DeepSpeed-Chat/training/step1_supervised_finetuning/main.py", line 356, in main() File "/home/rlhf/DeepSpeedExamples/applications/DeepSpeed-Chat/training/step1_supervised_finetuning/main.py", line 220, in main model = create_hf_model(AutoModelForCausalLM, File "/home/rlhf/DeepSpeedExamples/applications/DeepSpeed-Chat/training/utils/model/model_utils.py", line 49, in create_hf_model model.resize_token_embeddings(int( File "/home/anaconda3/envs/rl/lib/python3.9/site-packages/transformers/modeling_utils.py", line 1395, in resize_token_embeddings model_embeds = self._resize_token_embeddings(new_num_tokens) File "/home/anaconda3/envs/rl/lib/python3.9/site-packages/transformers/modeling_utils.py", line 1411, in _resize_token_embeddings self.set_input_embeddings(new_embeddings) File "/home/anaconda3/envs/rl/lib/python3.9/site-packages/transformers/modeling_utils.py", line 1236, in set_input_embeddings base_model.set_input_embeddings(value) File "/home/anaconda3/envs/rl/lib/python3.9/site-packages/transformers/modeling_utils.py", line 1238, in set_input_embeddings raise NotImplementedError NotImplementedError 请问这是怎么解决的呢

在glm的源码中加入一个set input embedding就好了