search

daquexian / onnx-simplifier

Simplify your onnx model
Apache License 2.0
3.85k stars 383 forks source link

[BUG] failed when the model size is larger than 2GB #262

Open luchangli03 opened 1 year ago

luchangli03 commented 1 year ago

Describe the bug failed when the model size is larger than 2GB

[libprotobuf ERROR /project/third_party/onnx-optimizer/third_party/protobuf/src/google/protobuf/message_lite.cc:449] onnx.ModelProto exceeded maximum protobuf size of 2GB: 3438405053 Traceback (most recent call last): File "convert_init_to_half.py", line 23, in onnx_model, check = simplify(onnx_model) File "/root/.local/lib/python3.8/site-packages/onnxsim/onnx_simplifier.py", line 194, in simplify check_ok = model_checking.compare( File "/root/.local/lib/python3.8/site-packages/onnxsim/model_checking.py", line 158, in compare onnx.checker.check_model(model_opt) File "/root/miniconda3/envs/tf2/lib/python3.8/site-packages/onnx/checker.py", line 119, in check_model C.check_model(protobuf_string, full_check) onnx.onnx_cpp2py_export.checker.ValidationError: The model does not have an ir_version set properly.

Si-XU commented 1 year ago

I have the same prob. Can anyone fix it?

wuxiaojingjing commented 1 year ago

When onnx serialises the model into a string, it uses protobuf, which, if the version is high, will make the serialised string wrong, and Onnx can't import it

pip install protobuf==3.20.3