search

KhronosGroup / NNEF-Tools

The NNEF Tools repository contains tools to generate and consume NNEF documents
https://www.khronos.org/nnef
222 stars 57 forks source link

Regarding input format #84

Closed hansely closed 5 years ago

hansely commented 5 years ago

In the NNEF Model Zoo, some of the models get input as nhwc while some models get input as nchw format. I assume that models that are converted from tensorflow is in nhwc format. Can you make this uniform? For example, when I look at the inceptionv3 & inceptionv4 graphs, it gets an input as 1,299,299,3 (nhwc), transpose it to nchw, run all the other operations as nchw, and transpose it back to nhwc at the end. The problem is, when I run the model with regular input images, it is not getting the expected results due to the the transpose layer at the beginning while other models are working fine. From my understanding, openvx graphs are in nchw format. Can there be a fix so that when tensorflow models are converted to NNEF, the input and output format will be fixed to nchw?

gyenesvi commented 5 years ago

Yes, you are right, now the models are converted to keep the input format of the original model. It might make sense to unify them (convert to NCHW), we need to discuss this, but the converter does have an option to do so (the help of the command line tool tells you how to parameterize it).

If you need such models quickly, you can re-convert some models for yourself, but unifying the model zoo would take some time for us.

hansely commented 5 years ago

That is fine. I can wait for it. Thanks.

gyenesvi commented 5 years ago

We have unified this in the model zoo, can you test it?

hansely commented 5 years ago

The graphs(inceptionv3, inceptionv4) looks fine but when I run the image classification, it it not showing the right answers unlike other models. It it just because the model is not trained well enough?

gyenesvi commented 5 years ago

I don't know, is it never showing good results (not even top 5), or is it just wrong in some cases? Did you test this before the unification of the input formats? Was it working then?

hansely commented 5 years ago

The way I tested is through AMD's MIVisionX classifer app: https://github.com/kiritigowda/MIVisionX-Inference-Tutorial which worked fine for other models. The models that are not giving right answers are googlenet, inceptionv1, v2, v3, and v4.

gyenesvi commented 5 years ago

Kiriti told me that he experienced something similar with ONNX models downloaded from the ONNX model zoo, if I understood correctly. Maybe he could tell if these are the same models.

hansely commented 5 years ago

Alright. Thanks for the help. Since the models are unified, I will close the issue.

gyenesvi commented 5 years ago

We have tested these networks, and as you say they don't seem to give good results, but we compared their output to the original ONNX/TensorFlow versions (in TF/Caffe2) and those give equally poor results, so it seems that the problem is with the original models, and not the conversion.

hansely commented 5 years ago

Did you try pre-processing? Some networks such as VGG16&19 needed preprocessing in order to get the correct results.

hansely commented 5 years ago

Here's the example of resnet:

https://github.com/onnx/models/tree/master/models/image_classification/resnet#preprocessing

gyenesvi commented 5 years ago

We have tried this normalisation for resent and it works on our side, giving good classification results. Also, we figured that for inception v3 and v4, normalising inputs to the range [-1,1] gives reasonable results for us (just mapping [0,255] to [-1,1], no mean subtraction or division). Also note that coloraturas format is RGB. For inception v1, range [0,255] with BGR coloraturas format works for us.