search

intel-analytics / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, GraphRAG, DeepSpeed, Axolotl, etc
Apache License 2.0
6.65k stars 1.26k forks source link

glm4:AttributeError: 'NoneType' object has no attribute 'shape' #11714

Open jjzhu0579 opened 2 months ago

jjzhu0579 commented 2 months ago

python3.11 transformers4.42.4

jjzhu0579 commented 2 months ago

Traceback (most recent call last): File "/share/home/aim/aim_zhujj/bc2/glm4_lora_train.py", line 138, in trainer.train() File "/data/aim_nuist/aim_zhujj/.conda/envs/blurb/lib/python3.11/site-packages/transformers/trainer.py", line 1938, in train return inner_training_loop( ^^^^^^^^^^^^^^^^^^^^ File "/data/aim_nuist/aim_zhujj/.conda/envs/blurb/lib/python3.11/site-packages/transformers/trainer.py", line 2279, in _inner_training_loop tr_loss_step = self.training_step(model, inputs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/aim_nuist/aim_zhujj/.conda/envs/blurb/lib/python3.11/site-packages/transformers/trainer.py", line 3318, in training_step loss = self.compute_loss(model, inputs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/aim_nuist/aim_zhujj/.conda/envs/blurb/lib/python3.11/site-packages/transformers/trainer.py", line 3363, in compute_loss outputs = model(inputs) ^^^^^^^^^^^^^^^ File "/data/aim_nuist/aim_zhujj/.conda/envs/blurb/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/aim_nuist/aim_zhujj/.conda/envs/blurb/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/aim_nuist/aim_zhujj/.conda/envs/blurb/lib/python3.11/site-packages/peft/peft_model.py", line 1625, in forward return self.base_model(inputs_embeds=inputs_embeds, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/aim_nuist/aim_zhujj/.conda/envs/blurb/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/aim_nuist/aim_zhujj/.conda/envs/blurb/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/share/home/aim/aim_zhujj/.cache/huggingface/modules/transformers_modules/glm-4-9b-chat/modeling_chatglm.py", line 878, in forward transformer_outputs = self.transformer( ^^^^^^^^^^^^^^^^^ File "/data/aim_nuist/aim_zhujj/.conda/envs/blurb/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/aim_nuist/aim_zhujj/.conda/envs/blurb/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/share/home/aim/aim_zhujj/.cache/huggingface/modules/transformers_modules/glm-4-9b-chat/modeling_chatglm.py", line 757, in forward batch_size, seq_length = input_ids.shape ^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'shape'

Uxito-Ada commented 2 months ago

Hi @jjzhu0579 ,

It seems that there is a ChatGLM-fine-tuneing application wrote by yourself, named glm4_lora_train.py, while we have provided LoRA fine-tuning for ChatGLM on IPEX-LLM. Pls follow here to try it. And currently, we do not support transformers 4.42.4 in ChatGLM fine-tuning.

jjzhu0579 commented 2 months ago

Hi @jjzhu0579 ,

It seems that there is a ChatGLM-fine-tuneing application wrote by yourself, named glm4_lora_train.py, while we have provided LoRA fine-tuning for ChatGLM on IPEX-LLM. Pls follow here to try it. And currently, we do not support transformers 4.42.4 in ChatGLM fine-tuning.

sorry ，the name is wrong,it is the p-tuning

jjzhu0579 commented 2 months ago

image

Uxito-Ada commented 2 months ago

Hi @jjzhu0579 ,

Sorry that p-tuning has not been supported. You can find all available fine-tuning algorithms here. In addition, IPEX-LLM works only on Intel XPU rather than CUDA.