Open rahulmate opened 11 months ago
The same error occurs even when no batching is used
sudo docker run -it --rm --gpus all \
-v $PWD/models:/project ghcr.io/els-rd/transformer-deploy:0.6.0 \
bash -c "pip3 install \".[GPU]\" && cd /project && \
convert_model -m \"sentence-transformers/multi-qa-mpnet-base-dot-v1\" \
--backend onnx \
--task embedding \
--seq-len 16 128 128"
Error:
Traceback (most recent call last):
File "/usr/local/bin/convert_model", line 8, in <module>
sys.exit(entrypoint())
File "/usr/local/lib/python3.8/dist-packages/transformer_deploy/convert.py", line 574, in entrypoint
main(commands=args)
File "/usr/local/lib/python3.8/dist-packages/transformer_deploy/convert.py", line 448, in main
[
File "/usr/local/lib/python3.8/dist-packages/transformer_deploy/convert.py", line 449, in <listcomp>
optimize_onnx(
File "/usr/local/lib/python3.8/dist-packages/transformer_deploy/backends/ort_utils.py", line 117, in optimize_onnx
optimized_model: BertOnnxModel = optimizer.optimize_model(
File "/usr/local/lib/python3.8/dist-packages/onnxruntime/transformers/optimizer.py", line 253, in optimize_model
optimizer = optimize_by_fusion(model, model_type, num_heads, hidden_size, optimization_options)
File "/usr/local/lib/python3.8/dist-packages/onnxruntime/transformers/optimizer.py", line 153, in optimize_by_fusion
optimizer.optimize(optimization_options)
File "/usr/local/lib/python3.8/dist-packages/onnxruntime/transformers/models/gpt2/../../onnx_model_bert.py", line 352, in optimize
self.fuse_reshape()
File "/usr/local/lib/python3.8/dist-packages/onnxruntime/transformers/models/gpt2/../../onnx_model_bert.py", line 77, in fuse_reshape
fusion.apply()
File "/usr/local/lib/python3.8/dist-packages/onnxruntime/transformers/models/gpt2/../../fusion_base.py", line 46, in apply
self.fuse(node, input_name_to_nodes, output_name_to_node)
File "/usr/local/lib/python3.8/dist-packages/onnxruntime/transformers/models/gpt2/../../fusion_reshape.py", line 171, in fuse
self.replace_reshape_node(shape, reshape_node, concat_node)
File "/usr/local/lib/python3.8/dist-packages/onnxruntime/transformers/models/gpt2/../../fusion_reshape.py", line 21, in replace_reshape_node
shape_value = np.asarray(shape, dtype=np.int64)
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (4,) + inhomogeneous part.
free(): invalid pointer
When converting the model I am getting below error. Command used.
Error