search

Samsung / ONE

On-device Neural Engine
Other
437 stars 158 forks source link

Compiler FE: support Shape op in luci-interpreter #5387

Open mhs4670go opened 3 years ago

mhs4670go commented 3 years ago

What

We need to support Shape op in luci-interpreter.

Why

When we test release binary, one of those model we run is inception_v3. You can get it with below command.

wget https://storage.googleapis.com/download.tensorflow.org/models/tflite/model_zoo/upload_20180427/inception_v3_2018_04_27.tgz
tar zxvf inception_v3_2018_04_27.tgz

There is no problem with tensorflow 1.x version but with tensorflow 2.x it's got different result.

./one-import-tf -i inception_v3.pb -o inception_v3.circle \
-I input -S 1,299,299,3 -O InceptionV3/Predictions/Reshape_1 \
--v2

image

Since the network has Shape operator, it throws NYI error when I run record-minmax with it.

I'd appreciate if you could support this feature.

jinevening commented 3 years ago

The problem stems from the dynamic shape API in TFLite.

TF 2.X automatically generates those weird operators (Squeeze, Reshape, Shape) to support dynamic shape in TFLite. This is somewhat strange because we've given the static input shape to one-import-tf (-S 1,299,299,3 in the above example).

I've found out that currently tf2tfliteV2.py ignores input_shape option if it is called with --v2. I guess this is because there is no option to specify input shape in TF 2.0. Preferred workaround is to directly set shape in the graph. https://github.com/tensorflow/tensorflow/issues/30180#issuecomment-505959220

I've checked that the shape-changing operators (squeeze, reshape, shape) are not generated if the workaround is used. So, how about adding the workaround to tf2tfliteV2.py ? Maybe I can make a change for that.

jinevening commented 3 years ago

@mhs4670go #5394 may have solved the problem of this issue. Do you still want to implement SHAPE Op in luci-interpreter?

seanshpark commented 3 years ago

5394 may have solved the problem of this issue.

For TF models, it will work. What about tflite files that wasn't converted by our tool?

jinevening commented 3 years ago

What about tflite files that wasn't converted by our tool?

The official inception v3 tflite model does not have SHAPE Op, so it may not fail.

For other models with SHAPE Op, circle-quantizer may fail. I'll support it. Thanks.