search

Xilinx / finn

Dataflow compiler for QNN inference on FPGAs
https://xilinx.github.io/finn
BSD 3-Clause "New" or "Revised" License
684 stars 219 forks source link

Extend support for the resize operator #1125

Open klassen9 opened 2 weeks ago

klassen9 commented 2 weeks ago

The PR extends/fixes the current implementations for the transformations of the resize operator in FINN.

We wanted to streamline a model that uses a resize operator, but the current implementation only supports resize via the scales input. The model we used, uses the sizes input. scales and sizes are both responsible for determining the amount of resizing and can be translated into each other. They are also mutually exclusive.

A problem that arose while implementing the support for sizes was to determine which input is used. Different versions of the Onnx opset have a different understanding of how to mark an input as unused. As an example:

I am thus checking if scales or sizes exist by checking if they are None or are empty to support all versions. None is returned by model.get_initializer() if the input is an empty string in the input list.

Furthermore, this PR adds support for the Version 10 resize operation. The Version 10 resize operation only supports the scales input.

Other than the scales and sizes inputs and the input tensor, there is the roi (region of interest) input. The roi is not supported by the hardware implementation and therefore triggers an assert in convert_to_hw_layers.py when used.

The test_scale_resize_nhwc.py test was extended by a test that uses the sizes input instead of the scales input.

I also published a PR for qonnx adding the resize operation to the change_3d_tensors_to_4d transformation (https://github.com/fastmachinelearning/qonnx/pull/125).

klassen9 commented 1 week ago

Sorry, I didn't apply pre-commit correctly. Should be fixed in the second commit.