Converter error: list assignment index out of range

[anilsathyan7]

I was trying to convert a pytorch model into keras via onnx. The pytorch model was successfully converted to onnx format. I tried two versions of the model:

1. Model with bilinear upsampling, with align_corners=False and opset 9,
2. Model with bilinear upsampling, with align_corners=True and opset 11

In both cases i got same errors, if i set change_ordering=False Code:-

import onnx
from onnx2keras import onnx_to_keras

# Load ONNX model
onnx_model = onnx.load('SINet_320_opbnup.onnx')

# Call the converter (input - is the main model input name, can be different for your model)
k_model = onnx_to_keras(onnx_model, ['data'])
k_model.save('SINet.h5')

Error:-

WARNING:onnx2keras:avgpool:Unable to use `same` padding. Add ZeroPadding2D layer to fix shapes.
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-27-b90de0dc63bc> in <module>()
      6 
      7 # Call the converter (input - is the main model input name, can be different for your model)
----> 8 k_model = onnx_to_keras(onnx_model, ['data'])
      9 k_model.save('SINet.h5')

6 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/layers/advanced_activations.py in build(self, input_shape)
    141     if self.shared_axes is not None:
    142       for i in self.shared_axes:
--> 143         param_shape[i - 1] = 1
    144     self.alpha = self.add_weight(
    145         shape=param_shape,

IndexError: list assignment index out of range

If i set change_ordering=True, i get another error:-

---

ValueError Traceback (most recent call last)

in () 6 7 # Call the converter (input - is the main model input name, can be different for your model) ----> 8 k_model = onnx_to_keras(onnx_model, ['data'], change_ordering=True) 9 k_model.save('SINet.h5') 5 frames /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py in set_weights(self, weights) 1824 raise ValueError( 1825 'Layer weight shape %s not compatible with provided weight ' -> 1826 'shape %s' % (ref_shape, weight.shape)) 1827 weight_value_tuples.append((param, weight)) 1828 weight_index += 1 ValueError: Layer weight shape (1, 1, 160) not compatible with provided weight shape (12, 1, 1)

Models:-

SINet_320.zip

I ran the model checker successfully i.e onnx.checker.check_model(onnx_model) The original pytorch model contains PRelu activations. How can we fix this issue and convert the model into keras format??

[Andredance]

Hi! Сan you check if this is what you were looking for? SINet_320_opbnup.h5.zip

[anilsathyan7]

The structure seems to be fine; but how to load this model with channel first nodes and lambda layers in tf.keras inference?

I got the following error during load:-

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/layers/core.py:1045: UserWarning: onnx2keras.convolution_layers is not loaded, but a Lambda layer uses it. It may cause errors. , UserWarning)

ValueError Traceback (most recent call last)

in () 2 import tensorflow as tf 3 ----> 4 model=tf.keras.models.load_model('/content/SINet_320_opbnup.h5') 12 frames /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/utils/generic_utils.py in func_load(code, defaults, closure, globs) 455 except (UnicodeEncodeError, binascii.Error): 456 raw_code = code.encode('raw_unicode_escape') --> 457 code = marshal.loads(raw_code) 458 if globs is None: 459 globs = globals() ValueError: bad marshal data (unknown type code)

Ideally, we want NHWC format with bilinear upsample, so that we can deploy them in tflite format ...

[Andredance]

Ohh. Unfortunately, it turned out to be not as easy as I thought. I can highlight the following problems:

• bilinear mode of Upsample operation is not supported by onnx2keras. Onnx graph specify mode as linear (in theory, it is possible to use a keras layer with a bilinear mode, but since it is not determined well by Onnx, we can't add it). So the only way is to customize convert_upsample function by yourself.
• ReduceMax operation is not supported yet (I created a PR, but I can't say if it will be approved)
• Prelu need a fix too, which is also in PR status
• This architecture during conversion will use multiple lambda layers, which probably won't work fine with change_ordering=True (these are only thoughts)

[anilsathyan7]

I think the 'resize' operator in onnx seems to handle bilinear resize functionality. It seems to give good results during testing/inference, with align corners=True. Earlier it used to be upsample layer(ie. nearest only). Similarly onnx seems to support reduce max operation. Keras also seems to have these layers anyway, so are they just integration issues?

[Andredance]

I think this is primarily a lack of time. ReduceMax operation was added yesterday. I will look better to onnx "Resize" operation and maybe will add its support to the library. There is no similar functionality in keras. I'll see what can be done about it.