search

onnx / models

A collection of pre-trained, state-of-the-art models in the ONNX format
http://onnx.ai/models/
Apache License 2.0
7.85k stars 1.4k forks source link

Image classification models don't support batching, batch size fixed at 1 #233

Open deadeyegoodwin opened 4 years ago

deadeyegoodwin commented 4 years ago

For example, if I look at the resnet50 model I see an input shape like this: [ 1, 3, 224, 224 ]. Why is the batch dimension fixed at 1 instead of being -1? Batching often provides large speedups when running these models so by limiting these examples to batch-size 1 you are not showing off the performance potential of ONNX (and runtimes like ONNXRuntime). Is it possible to regenerate or modify the existing models so that they support batching?

snnn commented 4 years ago

You may use tf2onnx converter and get the source models from https://github.com/tensorflow/models/tree/master/research/slim#pre-trained-models

deadeyegoodwin commented 4 years ago

@snnn Thanks for the pointer. But, just to be clear, I was asking if the models in the model zoo can be updated to support batching. The model zoo is much less useful if it doesn't contain models in the format that most people want... and I think for image classification models most people want models that support batching.