microsoft / MMdnn

MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.
MIT License
5.78k stars 968 forks source link

Fix parse bug #904

Open Sundrops opened 3 years ago

Sundrops commented 3 years ago

https://github.com/microsoft/MMdnn/issues/882

lxgyChen commented 3 years ago

@Sundrops 好像不起作用啊#882,还是报一样的错误。

Sundrops commented 3 years ago

It's ok on torch1.6.0

lxgyChen commented 3 years ago

我刚试过,在torch1.5.1中,当output_shape_str=['%500 : Float', '1, 256, 20, 20', ' ']时,在torch1.6.0中是['%500 : Float', '1:102400, 256:400, 20:20, 20:1', ' ``'],这种情况下在目前的mmdnn版本下也是能正确解析的。 但在我转换yolov5s的情况,见#882,输出的output_shape_str:

['%498 : Float', '1:102400, 256:400, 20:20, 20:1', ' '] ['%499 : Float', '1:102400, 256:400, 20:20, 20:1', ' '] ['%500 : Double', ' '] Traceback (most recent call last): File "./convertToIR.py", line 202, in <module> _main() File "./convertToIR.py", line 197, in _main ret = _convert(args) File "./convertToIR.py", line 97, in _convert parser = PytorchParser151(model, inputshape[0]) File "/home/user/anaconda3/envs/pytorch/lib/python3.7/site-packages/mmdnn/conversion/pytorch/pytorch_parser.py", line 533, in __init__ self.build_graph(input_shape) File "/home/user/anaconda3/envs/pytorch/lib/python3.7/site-packages/mmdnn/conversion/pytorch/pytorch_parser.py", line 92, in build_graph self.pytorch_graph.build(self.input_shape) File "/home/user/anaconda3/envs/pytorch/lib/python3.7/site-packages/mmdnn/conversion/pytorch/pytorch_graph.py", line 136, in build output_shape = [int(x.replace('!', '').split(':')[0]) for x in output_shape_str[1].split(',')] File "/home/user/anaconda3/envs/pytorch/lib/python3.7/site-packages/mmdnn/conversion/pytorch/pytorch_graph.py", line 136, in <listcomp> output_shape = [int(x.replace('!', '').split(':')[0]) for x in output_shape_str[1].split(',')] ValueError: invalid literal for int() with base 10: ' '

当output_shape_str=['%500 : Double', ' ']时,解析出来就是空字符串‘ ’了,转换为int型就报错了。这种情况怎么处理呢? 我把conversion/pytorch/pytorch_graph.py 大约在134行的

if len(output_shape_str) > 1 改成 if len(output_shape_str) > 2 不会报错了,但是网络没有完全转换成功,只转换了第一层,而且提示PyTorch parser has not supported operator [onnx::Constant]. IR network strucuture may lost info.

Sundrops commented 3 years ago

@lxgyChen do you use Upsample layer in yolov5s? Upsample layer will be converted to

, %224 : Double() = onnx::Constant[value={2}](), scope: CustomizedNet/Upsample[Sampling30]
, %225 : Double() = onnx::Constant[value={2}](), scope: CustomizedNet/Upsample[Sampling30]
, %226 : Tensor = onnx::Unsqueeze[axes=[0]](%224), scope: CustomizedNet/Upsample[Sampling30]
, %227 : Tensor = onnx::Cast[to=1](%226), scope: CustomizedNet/Upsample[Sampling30]
, %228 : Tensor = onnx::Unsqueeze[axes=[0]](%225), scope: CustomizedNet/Upsample[Sampling30]
, %229 : Tensor = onnx::Cast[to=1](%228), scope: CustomizedNet/Upsample[Sampling30]
, %230 : Tensor = onnx::Constant[value= 1  1 [ CPUFloatType{2} ]](), scope: CustomizedNet/Upsample[Sampling30]
, %231 : Tensor = onnx::Concat[axis=0](%230, %227, %229), scope: CustomizedNet/Upsample[Sampling30]
, %232 : Tensor = onnx::Constant[value=[ CPUFloatType{0} ]](), scope: CustomizedNet/Upsample[Sampling30]
, %233 : Float(1:60768, 422:144, 12:12, 12:1) = onnx::Resize[coordinate_transformation_mode="asymmetric", cubic_coeff_a=-0.75, mode="nearest", nearest_mode="floor"](%223, %232, %231), scope: CustomizedNet/Upsampl
e[Sampling30] # /mnt/WXRC0020/release/gluoncvplus/.local/opt/anaconda3/envs/gp_v0.0.1/lib/python3.7/site-packages/torch/nn/functional.py:3143:0

And mmdnn does not support Constant op.

lxgyChen commented 3 years ago

@Sundrops 上面的output_shape_str=['%500 : Double', ' ']还没到Upsample,但也是一个Constant。YOLO v3 v4 v5都有Upsample层,那这样都不能转换了啊?

Sundrops commented 3 years ago

@lxgyChen As mentioned above, Upsample layer will be converted to onnx:: Constant and other layers. If you want to use Upsample, you should modify code to merge onnx:: Constant, ... onnx::Resize in onnx graph(https://github.com/microsoft/MMdnn/blob/master/mmdnn/conversion/pytorch/pytorch_graph.py#L231)

lxgyChen commented 3 years ago

@Sundrops 好的,我研究一下。怎么避免Constant Op呢,用已知的固定值代替可以吗?PyTorch导出为ONNX时可以选择Constant Folding,但是MMdnn不支持ONNX to IR