search

epfLLM / Megatron-LLM

distributed trainer for LLMs
Other
529 stars 76 forks source link

RuntimeError: mat1 and mat2 shapes cannot be multiplied (29056x22016 and 11008x4096) #73

Closed liuxm117 closed 1 year ago

liuxm117 commented 1 year ago

image hi, can you help to see what could be the cause of this error？

kylematoba commented 1 year ago

We'll need more information about the model to be of any help. Please reopen with a MWE (preferable) or at least some details of what you're trying to do.

liuxm117 commented 1 year ago

i use tp=1, pp=2 for llama2 7B finetune，