search

Samsung / ONE

On-device Neural Engine
Other
429 stars 157 forks source link

[compiler] Support dynamic shape inference for Conv2D and DepthwiseConv2D #14115

Open pcs1265 opened 1 week ago

pcs1265 commented 1 week ago

Currently, we are working on enabling Yamnet to support dynamic shape inference for the Reshape operation.

In this process, we planned to infer all dimensions as unknown when the shape is not CircleConst.

According to our decision, the height, width, and number of channels will be provided to Conv2D in unknown state, resulting an incorrect inference (the unknown dim will be treated as 0).

Therefore, if the height, width, and channels of the IFM are unknown, Conv2D should infer those dimensions as unknown.

pcs1265 commented 1 week ago

So I am planning this in the following sequence:

  1. Migrate Conv2D / DepthwiseConv2D related functions to sinf::Algorithm.
  2. Implement dynamic shape support for Conv2D.
  3. Implement dynamic shape support for DepthwiseConv2D.

However, I am concerned about the migration process. https://github.com/Samsung/ONE/blob/06a536b634f08145f1656adc7664b8099776f9a6/compiler/luci/service/src/CircleShapeInferenceRule.cpp#L420 There is a function infer_conv2d_type() that is shared by Conv2D and DepthwiseConv2D. To migrate this, I have considered several ways to do this:

  1. Create HelperConv2Ds.h and include the implementation in that header file.
  2. Add the function header to CircleShapeInferenceHelper.h
    • Then the implementation will go to CircleShapeInferenceHelper.cpp
    • Explicit instantiation is necessary because infer_conv2d_type() is a template function.
  3. Follow option 2, but as a non-template function without instantiation.
    • The function will take 5 arguments (ifm_shape, ker_shape, stride, padding, dilation) -> which might be too many.

Which option do you prefer? I was working on first method.

seanshpark commented 1 week ago

Which option do you prefer? I was working on first method.

Please go one with your first method and lets see how it comes out.

pcs1265 commented 1 week ago

Which option do you prefer? I was working on first method.

Please go one with your first method and lets see how it comes out.

Thank you. I'll work on this way.