search

onnx / tutorials

Tutorials for creating and using ONNX models
Apache License 2.0
3.34k stars 626 forks source link

NotImplementedError: Unsupported ONNX ops of type: Shape,Gather,Cast,Floor #150

Open edjust opened 4 years ago

edjust commented 4 years ago

After the step of convert the ONNX models to CoreML models it appears this to me:

(venv) Edgars-MacBook-Pro-152:fast_neural_style edjust$ python onnx_to_coreml.py ./saved_models/candy.onnx ./saved_models/candy.mlmodel Traceback (most recent call last): File "onnx_to_coreml.py", line 11, in coreml_model = convert(model_proto, image_input_names=['0'], image_output_names=['186']) File "/Users/edjust/venv/lib/python3.7/site-packages/onnx_coreml/converter.py", line 610, in convert _check_unsupported_ops(graph.nodes, disable_coreml_rank5_mapping) File "/Users/edjust/venv/lib/python3.7/site-packages/onnx_coreml/converter.py", line 222, in _check_unsupported_ops ','.join(unsupported_op_types), coreml_3_rerun_message)) NotImplementedError: Unsupported ONNX ops of type: Shape,Gather,Cast,Floor Please try converting again with target_ios=13 and coremltools 3.0 latest beta

I have already changed the version to 13 in the variable target_ios and downloaded the last version of CoreML. How could I resolve this?

bhushan23 commented 4 years ago

@edjust please use latest onnx-coreml beta https://github.com/onnx/onnx-coreml/#new-beta-onnx-coreml-converter-with-core-ml-3 pass target_ios='13' these ops are supported with target_ios='13'

edjust commented 4 years ago

@bhushan23 thank you very much... I downloaded the latest onnx-coreml from the link that you posted and replace it by the one that was at: /Users/edjust/venv/lib/python3.6/site-packages/onnx_coreml and I pass by that issue.

Now appears this to me after compile the same code:

(venv) edjust@Edgars-MacBook-Pro-152 fast_neural_style % python onnx_to_coreml.py ./saved_models/candy.onnx ./saved_models/candy.mlmodel 1/81: Converting Node Type Pad 2/81: Converting Node Type Conv 3/81: Converting Node Type InstanceNormalization 4/81: Converting Node Type Relu 5/81: Converting Node Type Pad 6/81: Converting Node Type Conv 7/81: Converting Node Type InstanceNormalization 8/81: Converting Node Type Relu 9/81: Converting Node Type Pad 10/81: Converting Node Type Conv 11/81: Converting Node Type InstanceNormalization 12/81: Converting Node Type Relu 13/81: Converting Node Type Pad 14/81: Converting Node Type Conv 15/81: Converting Node Type InstanceNormalization 16/81: Converting Node Type Relu 17/81: Converting Node Type Pad 18/81: Converting Node Type Conv 19/81: Converting Node Type InstanceNormalization 20/81: Converting Node Type Add 21/81: Converting Node Type Pad 22/81: Converting Node Type Conv 23/81: Converting Node Type InstanceNormalization 24/81: Converting Node Type Relu 25/81: Converting Node Type Pad 26/81: Converting Node Type Conv 27/81: Converting Node Type InstanceNormalization 28/81: Converting Node Type Add 29/81: Converting Node Type Pad 30/81: Converting Node Type Conv 31/81: Converting Node Type InstanceNormalization 32/81: Converting Node Type Relu 33/81: Converting Node Type Pad 34/81: Converting Node Type Conv 35/81: Converting Node Type InstanceNormalization 36/81: Converting Node Type Add 37/81: Converting Node Type Pad 38/81: Converting Node Type Conv 39/81: Converting Node Type InstanceNormalization 40/81: Converting Node Type Relu 41/81: Converting Node Type Pad 42/81: Converting Node Type Conv 43/81: Converting Node Type InstanceNormalization 44/81: Converting Node Type Add 45/81: Converting Node Type Pad 46/81: Converting Node Type Conv 47/81: Converting Node Type InstanceNormalization 48/81: Converting Node Type Relu 49/81: Converting Node Type Pad 50/81: Converting Node Type Conv 51/81: Converting Node Type InstanceNormalization 52/81: Converting Node Type Add 53/81: Converting Node Type Upsample 54/81: Converting Node Type Pad 55/81: Converting Node Type Conv 56/81: Converting Node Type InstanceNormalization 57/81: Converting Node Type Relu 58/81: Converting Node Type Shape 59/81: Converting Node Type Gather 60/81: Converting Node Type Mul 61/81: Converting Node Type Cast 62/81: Converting Node Type Floor 63/81: Converting Node Type Shape 64/81: Converting Node Type Gather 65/81: Converting Node Type Mul 66/81: Converting Node Type Cast 67/81: Converting Node Type Floor 68/81: Converting Node Type Unsqueeze 69/81: Converting Node Type Unsqueeze 70/81: Converting Node Type Concat 71/81: Converting Node Type Shape 72/81: Converting Node Type Slice 73/81: Converting Node Type Div 74/81: Converting Node Type Concat 75/81: Converting Node Type Upsample Traceback (most recent call last): File "onnx_to_coreml.py", line 11, in coreml_model = convert(model_proto, image_input_names=['0'], image_output_names=['186']) File "/Users/edjust/venv/lib/python3.6/site-packages/onnx_coreml/converter.py", line 621, in convert _convert_node_nd(builder, node, graph, err) File "/Users/edjust/venv/lib/python3.6/site-packages/onnx_coreml/_operators_nd.py", line 2343, in _convert_node_nd return converter_fn(builder, node, graph, err) File "/Users/edjust/venv/lib/python3.6/site-packages/onnx_coreml/_operators.py", line 1627, in _convert_upsample "This ONNX upsample layer has 'scales' provided as an input. CoreML upsample requires 'scales' as an attribute of the layer.") File "/Users/edjust/venv/lib/python3.6/site-packages/onnx_coreml/_error_utils.py", line 60, in unsupported_op_configuration self.rerun_suggestion) TypeError: Error while converting op of type: Upsample. Error message: This ONNX upsample layer has 'scales' provided as an input. CoreML upsample requires 'scales' as an attribute of the layer. Please try converting with higher target_ios. You can also provide custom function/layer to convert the model.

I have already try target_ios = '13.1', target_ios = '13.1.1' and target_ios = '13.1.2' but no one works. I read some about it in https://github.com/onnx/onnx-coreml/issues/365 but from what I see that change is made it already.

Screen Shot 2019-10-08 at 04 05 39
bhushan23 commented 4 years ago

This is known PyTorch -> ONNX conversion issue where scale is mapped into multiple ops. converting static upsample into dynamic upsample. Here's the workaround for now, https://github.com/onnx/onnx-coreml/issues/453#issuecomment-525540974 i.e. use custom_conversion_function for Upsample to map resize_bilinear with known scale (usually scale is known in pytorch land) or use size in your pytorch code instead of scale if you use size then onnx-coreml works out of the box

regarding target_ios highest one is 13 supported

edjust commented 4 years ago

@bhushan23 could you give more details about it please? From what I understand I need to use the following code in pytorch/torch/onnx/symbolic_opset9.py from the issue https://github.com/pytorch/pytorch/issues/27376 OR where do I have to use "size" instead of "scale":

import torch import torch.nn as nn

class TestModel(nn.Module): def forward(self, x): return torch.nn.functional.interpolate(x, mode='nearest', scale_factor=2)

model = TestModel() dummy_input = torch.randn(5,100,100) torch.onnx.export(model, dummy_input, "pytorch_upsample_scale.onnx")

class TestModel2(nn.Module): def forward(self, x): return torch.nn.functional.interpolate(x, mode='nearest', size=200)

model = TestModel2() torch.onnx.export(model, dummy_input, "pytorch_upsample_size.onnx")

bhushan23 commented 4 years ago

@edjust No. You have two options

  1. Provide size to Upsample / Interpolate instead of scale

  2. Use custom_conversion_function as follows

    
# Custom Function for UpSample
def _convert_upsample(builder, node, graph, err):
if 'scales' in node.attrs:
    scales = node.attrs['scales']
elif len(node.input_tensors):
    scales = node.input_tensors[node.inputs[1]]
else:
    # HACK: Manual scales
    # PROVIDE MANUAL SCALE HERE
    scales = [1, 1, 0.5, 0.5]

scale_h = scales[2]
scale_w = scales[3]
input_shape = graph.shape_dict[node.inputs[0]]
target_height = int(input_shape[-2] * scale_h)
target_width = int(input_shape[-1] * scale_w)

builder.add_resize_bilinear(
    name=node.name,
    input_name=node.inputs[0],
    output_name=node.outputs[0],
    target_height=target_height,
    target_width=target_width,
    mode='UPSAMPLE_MODE'
)

model = onnx.load_model('/path/to/onnx_model.onnx') coreml_model = convert(model, target_ios='13') #, custom_conversion_functions={'Upsample': _convert_upsample})

dzyjjpy commented 4 years ago

@bhushan23 Hi I solved errors about upsample and slice as described. And another issue occured. Here is the specific information: 344/361: Converting Node Type Gather 345/361: Converting Node Type Gather 346/361: Converting Node Type Gather 347/361: Converting Node Type Concat 348/361: Converting Node Type Concat 349/361: Converting Node Type TopK 350/361: Converting Node Type Slice Traceback (most recent call last): File "/home/jiapy/workspace/yolact/model_convert.py", line 223, in add_custom_layers=True, File "/home/jiapy/virtualEnv/py3.6torch1.2/lib/python3.6/site-packages/onnx_coreml/converter.py", line 626, in convert _convert_node_nd(builder, node, graph, err) File "/home/jiapy/virtualEnv/py3.6torch1.2/lib/python3.6/site-packages/onnx_coreml/_operators_nd.py", line 2443, in _convert_node_nd return converter_fn(builder, node, graph, err) File "/home/jiapy/virtualEnv/py3.6torch1.2/lib/python3.6/site-packages/onnx_coreml/_operators_nd.py", line 2010, in _convert_slice return _convert_slice_ir4v9(builder, node, graph, err) ## comment by jiapy 20200224, replaced by function below File "/home/jiapy/virtualEnv/py3.6torch1.2/lib/python3.6/site-packages/onnx_coreml/_operators_nd.py", line 1927, in _convert_slice_ir4v9 starts[current_axes] = ip_starts[i] IndexError: list assignment index out of range

Process finished with exit code 1

Dose it support TopK op right now? Thanks.

dzyjjpy commented 4 years ago

And I debug to confirm that the slice op node is empty for 350. I am confused whether the error is related to topK operation. @bhushan23 looking forward to your reply.

dzyjjpy commented 4 years ago

@bhushan23 Do you have some advice for this issue?