[cldnn] throw an error of "[IE] exception [clDNN] Mixed layouts of input tensors are not supported in fused eltwise"

[lisa0314]

After I use ngraph API to implement a mobilenetv1/v2 model, and I want to use inference engine to infer the model. When I try to create an ExecutableNetwork object, it throw an error - "[IE] exception [clDNN] Mixed layouts of input tensors are not supported in fused eltwise". However, the model created in the same way could run successfully on cpu(mkldnn) backend. I want to figure out why and when it will throw this type of error.

[vladimir-paramuzov]

@lisa0314 This seems to be a bug in fusion logic for eltwise node. Could you share your model so we can reproduce and fix the issue?

[mingmingtasd]

Resolved it. We find that when we use ngraph::Node to build a model, if the shapes of for eltwise node op::Add's two input tensors are different, the error will be thrown. For example, tensorA's shape is {1,3,224,224}, tensorB's shape is {3}, "tensorA + tensorB" will throw "[IE] exception [clDNN] Mixed layouts of input tensors are not supported in fused eltwise". We must reshape the tensorB to {1,3,1,1} to make its size equal to that of tensorA, than there will be no error. It seems that IE-clDNN will validate the shapes and specify the layouts according to the shapes. But for IE-MKLDNN, op::Add can accept two input tensors with different shapes.

[vladimir-paramuzov]

@mingmingtasd Have you used default broadcast rules (numpy) for op::Add node?

@lazarevevgeny Are eltwise inputs with shapes {1,3,224,224} and {3} valid? I have a feeling that {3} is not broadcastable to {1,3,224,224} with numpy rules.

[mingmingtasd]

Hi， Yes, the default broadcast will not give {1,3,224,224}. I have broadcasted by keeping {3} as the channel, and got {1,3,224,224} so that other plugins can run.

[vladimir-paramuzov]

I'm just trying to understand why ngraph accepts {1,3,224,224} and {3], and CPU plugin runs something, while such inputs seem to be inconsistent. So maybe some check is missing in graph validator for Add node.

[mingmingtasd]

I'm just trying to understand why ngraph accepts {1,3,224,224} and {3], and CPU plugin runs something, while such inputs seem to be inconsistent. So maybe some check is missing in graph validator for Add node.

Note that: Before adding these two tensor, I broadcast tensorB with shape {3} by adding a op::v1::Broadcast for it and set theaxes_mappingto keep the channel. So I can broadcast {3} to {1,3,224,224}. The default broadcasting should not succeed to do that.

[lazarevevgeny]

The {1,3,224,224} and {3} are not broadcastable and nGraph should not pass this case. Need to take a look at the IR to understand how this passed nGraph.

[jgespino]

Closing, feel free to re-open if additional assistance is needed.