Compiler FE: NCHW/NHWC layout for input/output nodes

Samsung / ONE

On-device Neural Engine

Other

437 stars 158 forks source link

Open seanshpark opened 3 years ago

seanshpark commented 3 years ago

We may need to provide some information about input/output node layout of first main subgraph.

Background:

Related with ONNX import: ONNX model default is NCHW for inputs, outputs
convert_nchw_to_nhwc others will try to remove redundant Ops such as transpose for performance
nchw_to_nhwc_input_shape will change input layout to NHWC
nchw_to_nhwc_output_shape will change output layout to NHWC

For nchw_to_nhwc_input_shape and nchw_to_nhwc_output_shape, there is no information saved to distinguish applied or not.

Let's solve this problem somehow.

seanshpark commented 3 years ago

Previous related issue: #5487

hseok-oh commented 3 years ago

We may need more field for data layout in circle schema

Data layout for image process operation (ex. conv2d, pool2d, etc)
- Channel first: CHW, NCHW, NCDHW
- Channel last: HWC, NHWC, NDHWC
Input & output type
- Image input: I/O data layout follows above data layout
- Raw: Not image input. Don't care layout

Maybe we can use current data_layout field for operation and introduce new field for I/O type.

seanshpark commented 3 years ago

Maybe we can use current data_layout field

Where can I see data_layout field ?

hseok-oh commented 3 years ago

Maybe we can use current data_layout field

Where can I see data_layout field ?

My typo: data_layout -> data_format