search

iree-org / iree

A retargetable MLIR-based machine learning compiler and runtime toolkit.
http://iree.dev/
Apache License 2.0
2.85k stars 620 forks source link

Support Resize Ops with Dynamic Width and Height #8148

Open mariecwhite opened 2 years ago

mariecwhite commented 2 years ago

Models with resize ops and dynamic width and height are failing to compile with this error:

Diagnostics:
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: 'tfl.pack' op PackOp: output tensor rank mismatch.
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: note: see current operation: %340 = "tfl.pack"(%337, %339) {axis = 0 : i32, values_count = 2 : i32} : (tensor<1xi32>, tensor<1xi32>) -> tensor<2xi32> loc("FeatureExtractor/MobilenetV2/fpn/top_down/nearest_neighbor_upsampling/ResizeNearestNeighbor/size"("/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite":0:0))
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: 'tfl.resize_nearest_neighbor' op ConvertResizeOp: resize dynamic input not supported.
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: note: see current operation: %341 = "tfl.resize_nearest_neighbor"(%294, %340) {align_corners = false, half_pixel_centers = false} : (tensor<1x?x?x128xf32>, tensor<2xi32>) -> tensor<1x?x?x128xf32> loc("FeatureExtractor/MobilenetV2/fpn/top_down/nearest_neighbor_upsampling/ResizeNearestNeighbor"("/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite":0:0))
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: 'tfl.pack' op PackOp: output tensor rank mismatch.
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: note: see current operation: %371 = "tfl.pack"(%368, %370) {axis = 0 : i32, values_count = 2 : i32} : (tensor<1xi32>, tensor<1xi32>) -> tensor<2xi32> loc("FeatureExtractor/MobilenetV2/fpn/top_down/nearest_neighbor_upsampling_1/ResizeNearestNeighbor/size"("/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite":0:0))
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: 'tfl.resize_nearest_neighbor' op ConvertResizeOp: resize dynamic input not supported.
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: note: see current operation: %372 = "tfl.resize_nearest_neighbor"(%350, %371) {align_corners = false, half_pixel_centers = false} : (tensor<1x?x?x128xf32>, tensor<2xi32>) -> tensor<1x?x?x128xf32> loc("FeatureExtractor/MobilenetV2/fpn/top_down/nearest_neighbor_upsampling_1/ResizeNearestNeighbor"("/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite":0:0))
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: 'tfl.pack' op PackOp: output tensor rank mismatch.
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: note: see current operation: %334 = "tfl.pack"(%331, %333) {axis = 0 : i32, values_count = 2 : i32} : (tensor<1xi32>, tensor<1xi32>) -> tensor<2xi32> loc("FeatureExtractor/MobilenetV2/fpn/top_down/nearest_neighbor_upsampling/ResizeNearestNeighbor/size"("/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite":0:0))
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: 'tfl.resize_nearest_neighbor' op ConvertResizeOp: resize dynamic input not supported.
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: note: see current operation: %335 = "tfl.resize_nearest_neighbor"(%288, %334) {align_corners = false, half_pixel_centers = false} : (tensor<1x?x?x128xf32>, tensor<2xi32>) -> tensor<1x?x?x128xf32> loc("FeatureExtractor/MobilenetV2/fpn/top_down/nearest_neighbor_upsampling/ResizeNearestNeighbor"("/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite":0:0))
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: 'tfl.pack' op PackOp: output tensor rank mismatch.
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: note: see current operation: %365 = "tfl.pack"(%362, %364) {axis = 0 : i32, values_count = 2 : i32} : (tensor<1xi32>, tensor<1xi32>) -> tensor<2xi32> loc("FeatureExtractor/MobilenetV2/fpn/top_down/nearest_neighbor_upsampling_1/ResizeNearestNeighbor/size"("/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite":0:0))
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: 'tfl.resize_nearest_neighbor' op ConvertResizeOp: resize dynamic input not supported.
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: note: see current operation: %366 = "tfl.resize_nearest_neighbor"(%344, %365) {align_corners = false, half_pixel_centers = false} : (tensor<1x?x?x128xf32>, tensor<2xi32>) -> tensor<1x?x?x128xf32> loc("FeatureExtractor/MobilenetV2/fpn/top_down/nearest_neighbor_upsampling_1/ResizeNearestNeighbor"("/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite":0:0))
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: 'tfl.shape' op : illegal op still exists
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: note: see current operation: %304 = "tfl.shape"(%271) : (tensor<1x?x?x128xf32>) -> tensor<4xi32> loc("FeatureExtractor/MobilenetV2/fpn/top_down/nearest_neighbor_upsampling/Shape"("/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite":0:0))
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: 'tfl.resize_nearest_neighbor' op : illegal op still exists
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: note: see current operation: %310 = "tfl.resize_nearest_neighbor"(%271, %309) {align_corners = false, half_pixel_centers = false} : (tensor<1x?x?x128xf32>, tensor<2xi32>) -> tensor<1x?x?x128xf32> loc("FeatureExtractor/MobilenetV2/fpn/top_down/nearest_neighbor_upsampling/ResizeNearestNeighbor"("/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite":0:0))
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: 'tfl.pack' op : illegal op still exists
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: note: see current operation: %335 = "tfl.pack"(%332, %334) {axis = 0 : i32, values_count = 2 : i32} : (tensor<1xi32>, tensor<1xi32>) -> tensor<2xi32> loc("FeatureExtractor/MobilenetV2/fpn/top_down/nearest_neighbor_upsampling_1/ResizeNearestNeighbor/size"("/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite":0:0))
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: 'tfl.pack' op : illegal op still exists
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: note: see current operation: %309 = "tfl.pack"(%306, %308) {axis = 0 : i32, values_count = 2 : i32} : (tensor<1xi32>, tensor<1xi32>) -> tensor<2xi32> loc("FeatureExtractor/MobilenetV2/fpn/top_down/nearest_neighbor_upsampling/ResizeNearestNeighbor/size"("/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite":0:0))
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: 'tfl.resize_nearest_neighbor' op : illegal op still exists
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: note: see current operation: %336 = "tfl.resize_nearest_neighbor"(%317, %335) {align_corners = false, half_pixel_centers = false} : (tensor<1x?x?x128xf32>, tensor<2xi32>) -> tensor<1x?x?x128xf32> loc("FeatureExtractor/MobilenetV2/fpn/top_down/nearest_neighbor_upsampling_1/ResizeNearestNeighbor"("/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite":0:0))
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: 'tfl.shape' op : illegal op still exists
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: note: see current operation: %312 = "tfl.shape"(%311) : (tensor<1x?x?x128xf32>) -> tensor<4xi32> loc("FeatureExtractor/MobilenetV2/fpn/top_down/Shape"("/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite":0:0))
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: 'tfl.strided_slice' op : illegal op still exists
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: note: see current operation: %342 = "tfl.strided_slice"(%336, %19, %341, %18) {begin_mask = 15 : i32, ellipsis_mask = 0 : i32, end_mask = 9 : i32, new_axis_mask = 0 : i32, shrink_axis_mask = 0 : i32} : (tensor<1x?x?x128xf32>, tensor<4xi32>, tensor<4xi32>, tensor<4xi32>) -> tensor<1x?x?x128xf32> loc("FeatureExtractor/MobilenetV2/fpn/top_down/strided_slice_5"("/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite":0:0))
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: 'tfl.strided_slice' op : illegal op still exists
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: note: see current operation: %316 = "tfl.strided_slice"(%310, %19, %315, %18) {begin_mask = 15 : i32, ellipsis_mask = 0 : i32, end_mask = 9 : i32, new_axis_mask = 0 : i32, shrink_axis_mask = 0 : i32} : (tensor<1x?x?x128xf32>, tensor<4xi32>, tensor<4xi32>, tensor<4xi32>) -> tensor<1x?x?x128xf32> loc("FeatureExtractor/MobilenetV2/fpn/top_down/strided_slice_2"("/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite":0:0))
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: 'tfl.shape' op : illegal op still exists
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: note: see current operation: %330 = "tfl.shape"(%317) : (tensor<1x?x?x128xf32>) -> tensor<4xi32> loc("FeatureExtractor/MobilenetV2/fpn/top_down/nearest_neighbor_upsampling_1/Shape"("/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite":0:0))
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: 'tfl.shape' op : illegal op still exists
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: note: see current operation: %338 = "tfl.shape"(%337) : (tensor<1x?x?x128xf32>) -> tensor<4xi32> loc("FeatureExtractor/MobilenetV2/fpn/top_down/Shape_1"("/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite":0:0))
/usr/local/google/home/mariewhite/github/iree-samples/tflitehub/tmp/ssd_mobilenet_v2_fpnlite_test.py/model.tflite:0:0: error: The following illegal operations still remain: 
    tfl.shape (count: 4)
    tfl.resize_nearest_neighbor (count: 2)
    tfl.pack (count: 2)
    tfl.strided_slice (count: 2)

Steps to reproduce: lit -v tflitehub/ssd_mobilenet_v2_fpnlite_test.py

mariecwhite commented 2 years ago

IR here

jpienaar commented 2 years ago

@NatashaKnk I think you worked on this one didn't you?

NatashaKnk commented 2 years ago

I worked mostly on dynamic shape support in the tosa->linalg lowering, if I'm not mistaken the issue here is in tfl->tosa. I'll pick it up though. @rsuderman I know you resolved similar issues before, any advice on where I should tackle this from?