InterleavedImageBatch: Support OnnxConstant data format

[hakanardo]

Thanx! Please consider adding a test in test_onnx2keras.py aswell.

[xsacha]

In my model, this came up in an area where I was returning results. The results were no longer 4 dimensional, so it failed here. It is my understanding we won't attempt transposing unless it is 4 dimensional tensor. The other conversions still use an assert to check dim length but I believe it should just be a no-op for this event.

[hakanardo]

If results are not 4-dim it should not be an InterleavedImageBatch. That data-type means that the data is a batch of interlieved-images and indicates that they are trasponsed versions of the corresponding tensor on the onnx side. So we should instead fix the layer that produces the non 4-dim InterleavedImageBatch tensor to return an tenser of a different type.

[xsacha]

That makes sense! I assume it is the op that reshapes the image then.

[hakanardo]

On a second thought: OnnxConstant means that the tensor is a compiletime constant and can be used in compile time constant propagations (i.e. calculations on it can be performed at compiletime). An InterlievedImageBatch contans data that varies runtime, so we cant convert it into an OnnxConstant. This has to be fixed in the calling op instead. Probably it only implements the compile time constant propagation path and not the runtime net generation path. Do you have some example net running into this issue?

[xsacha]

Here's an example ONNX model (weights are randomised) that hits this code path. FaceDetector.zip

When running this model on the camera, converted with this PR, I get the expected results.

[hakanardo]

Thanx! The issue seams to be the gather op. It's the very last op in your net and that's probably why your solution works for this particualr case. Do you have the pytorch code that produces that gather op?

[xsacha]

scores = F.softmax(classifications, dim=-1).select(2,1)

or

scores = F.softmax(classifications, dim=-1)[:,:,1]

which produces the ONNX graph:

  %629 = Softmax[axis = 2](%600)
  %630 = Constant[value = <Scalar Tensor []>]()
  %conf = Gather[axis = 2](%629, %630)

So softmax is run on the classification scores ( BxNx2 elements).

I am only interested in the second set of values from the scores, so I use: select(2,1) to grab BxN elements.

That's why I assumed slicing would be the best way to achieve the gather op here too. It wasn't working for some reason so I implemented it as tf.gather instead: https://github.com/AxisCommunications/onnx-to-keras/pull/21

[hakanardo]

OK, I think this is now resolved by I21e0a0795e005c6539b4b085b52b22cbba7babd0. Could you please verify?

[hakanardo]

Make that 53bce898d3dbc660473bc58f225fb4194b7be095 (current master).