search

philschmid / sagemaker-huggingface-llama-2-samples

86 stars 32 forks source link

RuntimeError: mat1 and mat2 shapes cannot be multiplied (4096x5120 and 1x2560) #3

Open monuminu opened 1 year ago

monuminu commented 1 year ago

Ran all the cells of Notebook to funetune LLama2 got this error.

  2023-07-20T16:08:06.067+05:30 return forward_call(*args, **kwargs) File "/opt/conda/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
  2023-07-20T16:08:06.068+05:30 output = old_forward(*args, **kwargs) File "/opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 408, in forward
  2023-07-20T16:08:06.068+05:30 hidden_states, self_attn_weights, present_key_value = self.self_attn( File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
  2023-07-20T16:08:06.068+05:30 return forward_call(*args, **kwargs) File "/opt/conda/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
  2023-07-20T16:08:06.068+05:30 output = old_forward(*args, **kwargs) File "/opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 295, in forward
  2023-07-20T16:08:06.068+05:30 query_states = [F.linear(hidden_states, query_slices[i]) for i in range(self.pretraining_tp)] File "/opt/conda/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 295, in
  2023-07-20T16:08:06.068+05:30 query_states = [F.linear(hidden_states, query_slices[i]) for i in range(self.pretraining_tp)]
  2023-07-20T16:08:06.068+05:30 RuntimeError: mat1 and mat2 shapes cannot be multiplied (4096x5120 and 1x2560)
philschmid commented 1 year ago

Did you make any changes? Did you make sure the requirements.txt is provided?