Hello,
I am trying to convert llama 2 7B model to mnn. I am using all the pip packages version as mentioned in requirements.txt with MNN version 2.8.1
I am getting the below error while converting the model
File "/Users/.cache/huggingface/modules/transformers_modules/Llama2_7B/modeling_llama.py", line 322, in forward
key_states = torch.cat([past_key_value[0], key_states], dim=2)
RuntimeError: Sizes of tensors must match except in dimension 2. Expected size 32 but got size 4 for tensor number 1 in the list.
Hello, I am trying to convert llama 2 7B model to mnn. I am using all the pip packages version as mentioned in requirements.txt with MNN version 2.8.1
I am getting the below error while converting the model
This error is triggered at this step https://github.com/wangzhaode/llm-export/blob/25fa2b033957d891691400c92be0ddacdde82c56/llm_export.py#L1013
The value of self.past_kv_shape = [32, 2, 1, 32, 0, 128] in Llama2_7b_Chat class
This code https://github.com/wangzhaode/llm-export/blob/25fa2b033957d891691400c92be0ddacdde82c56/llm_export.py#L256 Results in [2, 1, 32, 0, 128] as past_key_values and past_key_values[0] is indeed 2 from this logic used in modelling.py
How did you fix this issue?