ONNX gvsoc wrong results and zero cycles

[HaolinChen]

I successfully quantized my onnx model with nntool 8 bit quantization and run it on GVSOC.

Entering main controller
hp_size=320,wh_size=640, id_size=40960,regs_size=640.
Output allocation done!
bbox allocation done!
Cluster open done!
Network Constructor is OK!
Task allcation done!
Entering Loop!
=== Task setup
===Run Fairtrack Backbone===

However, I get wrong featuremap output. Besides, the cycles are also strange.

S142_MatAdd_32x8x10: Cycles:          0, Operations:       2560, Operations/Cycle: inf
                   S145_Conv2d_32x32x3x3_Relu: Cycles:          0, Operations:     739840, Operations/Cycle: inf
                   S148_Conv2d_32x32x3x3_Relu: Cycles:          0, Operations:     739840, Operations/Cycle: inf
                           S150_Op_Resize_109: Cycles:          0, Operations:      61440, Operations/Cycle: inf
                           S152_Op_Resize_116: Cycles:          0, Operations:      61440, Operations/Cycle: inf
                         S119_MatAdd_16x16x20: Cycles:          0, Operations:       5120, Operations/Cycle: inf
                   S126_Conv2d_16x16x3x3_Relu: Cycles:          0, Operations:     742400, Operations/Cycle: inf
                         S151_MatAdd_16x16x20: Cycles:          0, Operations:       5120, Operations/Cycle: inf
                   S155_Conv2d_16x16x3x3_Relu: Cycles:          0, Operations:     742400, Operations/Cycle: inf
                         S156_MatAdd_16x16x20: Cycles:          0, Operations:       5120, Operations/Cycle: inf
                   S159_Conv2d_16x16x3x3_Relu: Cycles:          0, Operations:     742400, Operations/Cycle: inf
                  S162_Conv2d_256x16x3x3_Relu: Cycles:          0, Operations:   11878400, Operations/Cycle: inf
                         S166_Conv2d_1x64x1x1: Cycles:          0, Operations:      20480, Operations/Cycle: inf
                         S170_Conv2d_2x64x1x1: Cycles:          0, Operations:      40960, Operations/Cycle: inf
                         S178_Conv2d_2x64x1x1: Cycles:          0, Operations:      40960, Operations/Cycle: inf
                       S174_Conv2d_128x64x1x1: Cycles:          0, Operations:    2621440, Operations/Cycle: inf

                                        Total: Cycles:          0, Operations:   51900800, Operations/Cycle: inf

They are all zeros. I want to know how can I solve this problem. Is it caused by Makefile configuration or nntool script? Or it's just autotiler or gvsoc bug. Actually, in nntool, I can get the right model output even the model is quantized.

[HaolinChen]

I have solved the zero cycle problem. It truns out that I didn't start timer.

#ifdef PERF
    printf("Start timer\n");
    gap_cl_starttimer();
    gap_cl_resethwtimer();
#endif

But I still get wrong results. NNTOOL can output the right featuremap, but the output of Autotiler is not right. It's very strange. I've used the newest SDK, but the problem still exits.

[marco-fariselli]

HI Haolin, you mean that the quantized execution in NNTool is correct but in the real target (the code generated by the autotiler that runs on gvsoc/board) doesn't match with NNTool? If it is the case please check you are providing the input data in the correct format in the real target, i.e. the quantized version of the data. While in NNTool you need to run on float input data, in the real target the model is fully quantized and requires data in the quantized version. You can check the quantized range/scaling factors of the inputs with G.qshow().