pytorch / glow

Compiler for Neural Network hardware accelerators
Apache License 2.0
3.21k stars 689 forks source link

Support for models without batch (3D inputs). #5485

Open vuzelac-cadence opened 3 years ago

vuzelac-cadence commented 3 years ago

For image processing, customers sometimes use models w/o batch (3D image) and we would like to add support for this. In our case, all the models always come with Unsqueeze at the inputs. There are two approaches:

  1. Introduce HWC and CHW layouts that extend current NHWC and NCHW layouts which allow users to specify the 3D input. This adds unecessary options just to deal with the fact that the model is loaded after the images.

  2. Swap model loading and input tensors loading order. As is, images are loaded first that actually might be considered as a wrong approach (?). With the model loaded first, the input placeholder type might be used as a check when loading images.

We would like to upstream this thus want to choose something after taking some input (@jfix71 )

jfix71 commented 3 years ago

What do you mean by images being loaded before the model? As in, we take the input shapes and types based on the provided input images and use them to setup the placeholders? We don't need to necessarily take the input shape from the input images even if they're loaded first. But yeah it seems it's a better approach than adding HWC and CHW.