Specifing input shapes example

[wschin]

When converting models from Core ML, the batch size is unknown (variable-length) by default. To overwrite this setting, one can specify their own input shapes.

Consider MNIST.mlmodel downloaded at here. We can set batch size to 3 by running the following conversion code.

from onnxmltools.utils import visualize_model
from onnxmltools.convert.coreml.convert import convert
from onnxmltools.convert.common.data_types import FloatTensorType
from winmltools.utils import save_model
from winmltools.utils import save_text
from coremltools.models.utils import load_spec

coreml_model = load_spec('MNIST.mlmodel')

# Each input type is a tuple of (variable_name, variable_type). To find Core ML variable names
# and their types, you can print out coreml_model.description. If the considered Core ML
# variable, say coreml_model.description.input[0], is an image, you need to find out its color
# space value via printing coreml_model.description.input[0].type.imageType.colorSpace. 
# If the value of colorSpace is 10/20/30, the corresponding color space is 'GRAY'/'RGB'/'BGR'.
initial_type = (coreml_model.description.input[0].name, FloatTensorType(shape=[3, 1, 28, 28], color_space='GRAY')) # Other allowed color spaces are "RGB" and "BGR"
onnx_model = convert(coreml_model, initial_types=[initial_type])
# The produced ONNX model in text format
save_text(onnx_model, "mnist.onnx.txt")
# The produced ONNX model
save_model(onnx_model, 'mnist.onnx')
# Call a simple visualization tool
visualize_model(onnx_model)

Another example using BGR image input is show below

from onnxmltools.utils import visualize_model
from onnxmltools.convert.coreml.convert import convert
from onnxmltools.convert.common.data_types import FloatTensorType
from winmltools.utils import save_model
from winmltools.utils import save_text
from coremltools.models.utils import load_spec

coreml_model = load_spec('FNS-Candy.mlmodel')

# Set batch size to 1 instead of using variable-size batch
initial_type = (coreml_model.description.input[0].name, FloatTensorType(shape=[1, 3, 720, 720], color_space='BGR')) # Other allowed color spaces are "RGB" and "GRAY"
onnx_model = convert(coreml_model, initial_types=[initial_type])
save_text(onnx_model, "fns.onnx.txt")
save_model(onnx_model, 'fns.onnx')

visualize_model(onnx_model)

[another-pjohnson]

If you are just trying to change the batch size, I've created a PR #146 that lets you do just that. Using the initial_types didn't really work for me because the output shape wasn't being changed and when i tried to do infer_shapes on the model, it would blow up.

[wschin]

One problem of having default_batch_size is that we need to define what is a batch dimension. Although many frameworks assume that the first axis is batch, there is nothing preventing someone to use the second axis as batch. Could you share more details about the blowing up? I believe there might be a bug. Thanks.

[another-pjohnson]

Well, default_batch_size already exists in the Topology class and is set to 1. It seems to do exactly what I want -- convert "None/??" dimension to value of default_batch_size. Here is the minimal code that reproduces my error:

import keras
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation

from onnx import shape_inference, optimizer
import onnxmltools
from onnxmltools.convert.common.data_types import FloatTensorType

batch_size=10

model = Sequential()
model.add(Dense(512, activation='relu', input_shape=(784,), kernel_initializer='glorot_uniform'))
model.add(Dropout(0.2))
model.add(Dense(512, activation='relu', kernel_initializer='glorot_uniform'))
model.add(Dropout(0.2))
model.add(Dense(10, activation='softmax', kernel_initializer='glorot_uniform'))

in_shape = model.inputs[0].shape.as_list()
in_shape[0] = batch_size
out_shape = model.outputs[0].shape.as_list()
out_shape[0] = batch_size
initial_type = [(model.inputs[0].name, FloatTensorType(shape=in_shape)),
                (model.outputs[0].name, FloatTensorType(shape=out_shape))]
onnx_model = onnxmltools.convert_keras(model=model, name='mnist_simple', initial_types=initial_type)
passes = ['eliminate_identity', 'fuse_consecutive_transposes', 'eliminate_nop_transpose']
optimized_onnx = optimizer.optimize(onnx_model, passes)
inferred_onnx = shape_inference.infer_shapes(optimized_onnx)

Grabbing shapes and input/output names, matching up types all to force the shape seems really klunky. onnxmltools.convert_keras(..., default_batch_size=10) seems much easier and does exactly what I want -- and what I imagine most other people would want to do. If it's much more difficult than changing the batch size after creating the onnx model, i don't see why anyone would use the initial_types to do the same thing:

# fix up batch size after onnx_model constructed:
onnx_model.graph.input[0].type.tensor_type.shape.dim[0].dim_value = batch_size
onnx_model.graph.output[0].type.tensor_type.shape.dim[0].dim_value = batch_size

[wschin]

Thank you @Oewyn. That internal batch size was initially designed for Core ML, where batch axis is well-defined. For Core ML, its first axis is always batch size! I don't know if Keras has such a strong rule to identify batch axis in a tensor. Editing shapes after conversion is unsafe; for example, a model is trained with specifically batch size = 10 (10 observation points of a time series) for predicting the next observation. That model might always want to consume input with batch size = 10 to make sure the statistics are consistent to those in training phase.

[ghoshaw]

@wschin , I tried your code to convert coreml to onnx, but it says " ValueError: Unknown color space for tensor image__0", How to solve this? thanks