Optimization on valid ONNX models fail with obscure message

[siorpaes]

Bug Report

Some valid ONNX models fail during optimization phase. The checker is not reporting any problem on the model though.

System information

• OS Platform: Linux Ubuntu 18.04
• ONNX version 1.6.0
• Python version 3.7.6
• CMake version: 3.14.0
• Protobuf version: 3.9.1

Reproduction instructions

import onnx
import argparse
from onnx import optimizer
import onnx.utils
import os

# Parse arguments
parser = argparse.ArgumentParser(description='')
parser.add_argument("--input", required=True, help="Input ONNX model")
args = parser.parse_args()

# Load ONNX model
print("Processing model:", args.input)
model = onnx.load(args.input)
onnx.checker.check_model(model)

# Remove dangling nodes and save new model
optimized_model = optimizer.optimize(model, ['eliminate_deadend'])
outputFile = os.path.splitext(args.input)[0] + "-opt.onnx"
onnx.save(optimized_model, outputFile)

Example ONNX model triggering the the issue is attached. posenet_mobilenet_float_075_1_default_1.onnx.gz

Expected behavior

An optimized model is created

Actual output

python optimize_model.py   --input posenet_mobilenet_float_075_1_default_1.onnx 
Processing model: posenet_mobilenet_float_075_1_default_1.onnx
Traceback (most recent call last):
  File "optimize_model.py", line 26, in <module>
    optimized_model = optimizer.optimize(model, ['eliminate_deadend'])
  File "/home/daz/anaconda3/envs/atonn/lib/python3.7/site-packages/onnx/optimizer.py", line 55, in optimize
    optimized_model_str = C.optimize(model_str, passes)
IndexError: _Map_base::at

[zhenhuaw-me]

Thank you @siorpaes! And, for my local try regarding the original issue, I have the error message as below (using ONNX 1.8).

WARNING: ONNX Optimizer has been moved to https://github.com/onnx/optimizer.
All further enhancements and fixes to optimizers will be done in this new repo.
The optimizer code in onnx/onnx repo will be removed in 1.9 release.

Traceback (most recent call last):
  File "repro.py", line 15, in <module>
    opt = onnx.optimizer.optimize(model, ['eliminate_deadend'])
  File "/Users/wzh/workspace/onnx/onnx.git/onnx/optimizer.py", line 55, in optimize
    optimized_model_str = C.optimize(model_str, passes)
IndexError: Input MobilenetV1/Conv2d_0/weights is undefined!

For the error message I got here, it seems to me that, the ProtoBuf Graph reader graphProtoToGraph expects the weights (initializers) to be input of the graph? Here, the error indicates that MobilenetV1/Conv2d_0/weights should be included in value_by_name_of, which documented to included initializer but seems not in code?

[zhenhuaw-me]

Off the topic, per the Optimizer move warning, should we discuss here or move to the optimizer repo?

[askhade]

@jackwish @siorpaes moving this issue to the optimizer repo and it is the correct place for it.

[daquexian]

@siorpaes @jackwish @askhade https://github.com/onnx/onnx/pull/3007 or https://github.com/onnx/optimizer/pull/13 need to be merged to solve this issue. Before either of them gets merged, you can try https://github.com/daquexian/onnx-simplifier, which is a tool based on onnxoptimizer

[zhenhuaw-me]

@daquexian thank you for the update! I don't even know ONNX IR requires initializers to be graph inputs. This is should not be a blocker issue for me (@siorpaes correct me please if that is not the case for you :) ), but it would be great if we can close the gap.

[siorpaes]

Thanks @daquexian for looking into this. @jackwish not a blocker issue for me as well.

[daquexian]

It is fixed in version 0.1.2

[elvinagam]

I don't know if it is related but

I am running into this one which apparently does not show what I am missing:

opt_model = optimizer.optimize_model(
    'onnx/bert-base-cased.onnx',
    'bert', 
    num_heads=12,
    hidden_size=768,
    optimization_options=opt_options)

Error log:

AssertionError                            Traceback (most recent call last)
[<ipython-input-13-8f4ba115d338>](https://localhost:8080/#) in <module>
      4     num_heads=12,
      5     hidden_size=768,
----> 6     optimization_options=opt_options)

8 frames
[/usr/local/lib/python3.7/dist-packages/onnxruntime_tools/transformers/../symbolic_shape_infer.py](https://localhost:8080/#) in _add_suggested_merge(self, symbols, apply)
    156 
    157     def _add_suggested_merge(self, symbols, apply=False):
--> 158         assert all([(type(s) == str and s in self.symbolic_dims_) or is_literal(s) for s in symbols])
    159         symbols = set(symbols)
    160         for k, v in self.suggested_merge_.items():

AssertionError: