Open NALLEIN opened 4 years ago
I fixed the problem that relu with multiple outputs may caused the following ops can't find corresponding input tensor. Relu needs to be executed seperately and there are continuous relu in this model.
Besides , there are 6D tensors in this model I can not find a clear format description for 6D tensor.
And I can just run the model in polyfill but cann't get right result due to these tensors.
Do you have any suggestions on dealing with 6D tensor permute, and data reorder ?
@huningxin Do you have any suggestions in this problem?
@huningxin Any suggestions?
Generally WebNN uses channel-last layout, OpenVINO uses channel-first layout. Could you please share what are the shapes (convolution, reshape, permute) when you run it with webml-polyfill?
Generally WebNN uses channel-last layout, OpenVINO uses channel-first layout. Could you please share what are the shapes (convolution, reshape, permute) when you run it with webml-polyfill?
Yes, the tensor is shown in this figure. The model reshapes the channel dimension to 3 dimensions and then 3 separate channel dimensions was permuted.
Is this figure just for openvino model? I suppose WebNN uses NHWC layout, e.g. so the shape of Convolution should be [1, 360, 640, 72]
, is it?
Is this figure just for openvino model? I suppose WebNN uses NHWC layout, e.g. so the shape of Convolution should be
[1, 360, 640, 72]
, is it? Yes, this is openvino model and the shape of conv output should be[1, 360, 640, 720]
and reshape output should be[1, 360, 720, 8, 3, 3]
. But I have no idea how to permute and I try to permute using order[0, 1, 4, 2, 5, 3]
and[0, 4, 1, 5, 2, 3]
but seems doesn't work.
Did you try to permute [1, 360, 720, 8, 3, 3]
to [1, 360, 3, 640, 3, 8]
, then reshape to [1, 1080, 1920, 8]
?
Did you try to permute
[1, 360, 720, 8, 3, 3]
to[1, 360, 3, 640, 3, 8]
, then reshape to[1, 1080, 1920, 8]
?
Yes,I tried to permute [1, 360, 720, 8, 3, 3]
to[1, 360, 3, 640, 3, 8]
, or[1, 3, 360, 3, 640, 8]
then reshape to [1, 1080, 1920, 8]
.
I replace rehspape->permute->reshape with depthToSpace op:
case 'Reshape': {
let nextNode = graph.nodes[i + 1];
let next2Node = graph.nodes[i + 3];
if(nextNode && next2Node && nextNode.operator === 'Permute' &&
next2Node.operator === 'Reshape' )
{
const input = node.inputs[0];
console.log(`input shape: [${input.shape()}]`);
const blockSize = node.outputs[0].shape()[4];
inputs.push(this._getTensorId(input));
inputs.push(this._addScalarInt32(blockSize));
console.log(`blockSize ${blockSize}`);
const output = next2Node.outputs[0];
// Add outputs
const outDims = output.shape();
const outputType = {
type: this._getTypeCode(output.dataType()), dimensions: outDims
};
const outputId = this._addNamedOperand(output.graphId(), outputType);
outputs.push(outputId);
console.log(` output shape: [${outDims}]`);
i += 3;
console.log('Merge Reshape->Permute->Reshape into depthToSpace');
opCode = this._nn.DEPTH_TO_SPACE;
}
This can bypass the processing of 6D tensor but still can not get the correct result, reasons may be:
depthToSpace op cannot handle two const operations
The model requires some pre-processing of mean and variance
Add new super-resolution model from openvino. Here is the corresponding information of the model :
I tried to run these two models in polyfill, but I encountered the following problems during the model compilation stage:
The reason is that the Tensor corresponding to the input cannot be found in the .bin file when analysing the eltwise operation . This problem can be reproduced in this repo. I only made small changes to
OpenVINOModelImporter.js
and the problem seems to be in the weight file or whenOpenVINOModelImporter.js
calling the_addTensorOperands
function the corresponding Tensor was not resolved . Currently I can't locate the problem. I encountered this error in eltwise operation in both models,corresponding to the operation of name = "39" and the operation of name = "37" respectively.