🐞Describe the bug

Possibly related to to https://github.com/apple/coremltools/issues/756, https://github.com/apple/coremltools/issues/880, https://github.com/apple/coremltools/issues/970, https://github.com/apple/coremltools/issues/992
Encounter error when running an inference task with a converted mlmodel configured for flexible input + output shape.
Model converted from PyTorch.
No error during conversation, only runtime error.
Verified bug on both XCode & CoreMLTools prediction API

Trace/Error Message

XCode

# Code
let output = try? model!.prediction(image: pixelBuffer!)

# Trace
[espresso] [Espresso::handle_ex_plan] exception=Espresso exception: "Invalid argument": generic_general_concat_kernel: axis out of bounds status=-6
[coreml] Failure dynamically resizing for sequence length.
[coreml] Failure in resetSizes.

Using coremltools API

# Code 
output = model.predict({'image': img})

# Trace
Traceback (most recent call last):
  File "/Users/user/Documents/fast_nst/iphone_test.py", line 23, in <module>
    out_dict = model.predict({'image': img})
  File "/Users/user/miniconda3/envs/NST/lib/python3.8/site-packages/coremltools/models/model.py", line 329, in predict
    return self.__proxy__.predict(data, useCPUOnly)
RuntimeError: {
    NSLocalizedDescription = "Failure dynamically resizing for sequence length.";
}

To Reproduce [FB8985338 with Source File]

Convert a Pytroch model into CoreML Model & configure for flexible input/output [CODE BELOW]
Run prediction using coremltools API on the .mlmodel generated (In this case, make sure to test it with an image of size different from the one configured for the model).

I attached the code file in FB8985338 that contains:
- convert.py (Configure the model & Convert the model)
- inference.py (Use coremltools API to perform prediction)
- Model Weight File
- Model Source File

Below is the example code that performs the conversion & configuration for flexible input/output


# Configure output to be of ImaegType
def convert_multiarray_output_to_image(spec, feature_name, is_bgr=False):
from coremltools.proto import FeatureTypes_pb2 as ft
spec.description.output[0].type.multiArrayType.shape.append(3)
spec.description.output[0].type.multiArrayType.shape.append(HEIGHT)
spec.description.output[0].type.multiArrayType.shape.append(WIDTH)
for output in spec.description.output:
    if output.name != feature_name:
        continue
    if output.type.WhichOneof('Type') != 'multiArrayType':
        raise ValueError("%s is not a multiarray type" % output.name)
    array_shape = tuple(output.type.multiArrayType.shape)
    channels, height, width = array_shape
    if channels == 1:
        output.type.imageType.colorSpace = ft.ImageFeatureType.ColorSpace.Value('GRAYSCALE')
    elif channels == 3:
        if is_bgr:
            output.type.imageType.colorSpace = ft.ImageFeatureType.ColorSpace.Value('BGR')
        else:
            output.type.imageType.colorSpace = ft.ImageFeatureType.ColorSpace.Value('RGB')
    else:
        raise ValueError("Channel Value %d not supported for image inputs" % channels)
    output.type.imageType.width = width
    output.type.imageType.height = height

Configure spec to support flexible input & output image

def convert_flexible(spec): img_size_ranges = flexible_shape_utils.NeuralNetworkImageSizeRange() img_size_ranges.add_height_range((64, -1)) img_size_ranges.add_width_range((64, -1)) flexible_shape_utils.update_image_size_range(spec, feature_name=INPUT_NAME, size_range=img_size_ranges) flexible_shape_utils.update_image_size_range(spec, feature_name=OUTPUT_NAME, size_range=img_size_ranges)

net = Load_Model().eval() # Set in Eval mode! net.load_state_dict(torch.load(MODEL_WEIGHTS_PATH, map_location=torch.device(device))) net = net.to(device)

example_input = torch.rand(1, 3, HEIGHT, WIDTH) # Figure out the proper size traced_model = torch.jit.trace(net, example_input)

model_from_torch = ct.convert(traced_model, source="pytorch", inputs=[ct.ImageType(name=INPUT_NAME, shape=example_input.shape)]) model_from_torch = ct.models.MLModel(model_spec) model_from_torch.save(f"model.mlmodel")



## System environment (please complete the following information):
 - coremltools version  (e.g., 3.0b5): 4.0
 - OS (e.g., MacOS, Linux): MacOS Big Sur
 - macOS version (if applicable): 11.1
 - XCode version (if applicable): 12.4
 - How you install python (anaconda, virtualenv, system): miniconda
 - python version (e.g. 3.7): 3.8.5
 - any other relevant information:
     - torch==1.6.0

## Additional context
1. Submitted bug report with source file at FB8985338.

apple / coremltools

Converted Pytroch model can't support flexible Input/Output Shape for ImageType #1075