Ki6an / fastT5

⚡ boost inference speed of T5 models by 5x & reduce the model size by 3x.
Apache License 2.0
567 stars 72 forks source link

how to fix when convert model mT5 with max_length = 512 #44

Open batman-do opened 2 years ago

batman-do commented 2 years ago

2022-03-10 09:07:54.967587868 [W:onnxruntime:, execution_frame.cc:811 VerifyOutputSizes] Expected shape from model of {batch,sequence,2,64} does not match actual shape of {5,12,21,64} for output output_past_key_values 2022-03-10 09:07:54.969066346 [W:onnxruntime:, execution_frame.cc:811 VerifyOutputSizes] Expected shape from model of {1,12,2,64} does not match actual shape of {5,12,21,64} for output 566 2022-03-10 09:07:54.973768695 [W:onnxruntime:, execution_frame.cc:811 VerifyOutputSizes] Expected shape from model of {1,12,2,64} does not match actual shape of {5,12,21,64} for output 710 2022-03-10 09:07:54.978314803 [W:onnxruntime:, execution_frame.cc:811 VerifyOutputSizes] Expected shape from model of {1,12,2,64} does not match actual shape of {5,12,21,64} for output 854 2022-03-10 09:07:54.982609990 [W:onnxruntime:, execution_frame.cc:811 VerifyOutputSizes] Expected shape from model of {1,12,2,64} does not match actual shape of {5,12,21,64} for output 998 2022-03-10 09:07:54.986836355 [W:onnxruntime:, execution_frame.cc:811 VerifyOutputSizes] Expected shape from model of {1,12,2,64} does not match actual shape of {5,12,21,64} for output 1142 2022-03-10 09:07:54.991021618 [W:onnxruntime:, execution_frame.cc:811 VerifyOutputSizes] Expected shape from model of {1,12,2,64} does not match actual shape of {5,12,21,64} for output 1286 2022-03-10 09:07:54.995182350 [W:onnxruntime:, execution_frame.cc:811 VerifyOutputSizes] Expected shape from model of {1,12,2,64} does not match actual shape of {5,12,21,64} for output 1430 2022-03-10 09:07:54.999290875 [W:onnxruntime:, execution_frame.cc:811 VerifyOutputSizes] Expected shape from model of {1,12,2,64} does not match actual shape of {5,12,21,64} for output 1574 2022-03-10 09:07:55.003348281 [W:onnxruntime:, execution_frame.cc:811 VerifyOutputSizes] Expected shape from model of {1,12,2,64} does not match actual shape of {5,12,21,64} for output 1718 2022-03-10 09:07:55.007545242 [W:onnxruntime:, execution_frame.cc:811 VerifyOutputSizes] Expected shape from model of {1,12,2,64} does not match actual shape of {5,12,21,64} for output 1862 2022-03-10 09:07:55.011838101 [W:onnxruntime:, execution_frame.cc:811 VerifyOutputSizes] Expected shape from model of {1,12,2,64} does not match actual shape of {5,12,21,64} for output 2006

I have onnx==1.11.0 and onnxruntime==1.10.0

ZYshakalaka commented 2 years ago

I have the same problem

matthew-wei commented 2 years ago

def sequence_padding(inputs, length=None, padding=0): """Numpy函数,将序列padding到同一长度 """ if length is None: length = max([len(x) for x in inputs])

pad_width = [(0, 0) for _ in np.shape(inputs[0])]
outputs = []
for x in inputs:
    x = x[:length]
    pad_width[0] = (0, length - len(x))
    x = np.pad(x, pad_width, 'constant', constant_values=padding)
    outputs.append(x)

return torch.from_numpy(np.array(outputs, dtype='int64'))
ZYshakalaka commented 2 years ago

def sequence_padding(inputs, length=None, padding=0): """Numpy函数,将序列padding到同一长度 """ if length is None: length = max([len(x) for x in inputs])

pad_width = [(0, 0) for _ in np.shape(inputs[0])]
outputs = []
for x in inputs:
    x = x[:length]
    pad_width[0] = (0, length - len(x))
    x = np.pad(x, pad_width, 'constant', constant_values=padding)
    outputs.append(x)

return torch.from_numpy(np.array(outputs, dtype='int64'))

这些代码怎么用呀?