Instructions for quantization

[boomer319]

Hello,

I want to get the NanoFlowNet running on the AI deck for obstacle avoidance. The following shows an older version of nntool_script.

What were the pictures displaying (e.g. samples/img_OUT_7_a.ppm)? What are the images supposed to display? Since these are used for quantization, they should represent typical inputs for the neural network?

I have collected images from the crazyflie's wifi video stream. Following which rule would you group the images into pairs?

Which file format is needed? Does it have to be .ppm file format? I have looked at the nntools data_importer.py. Seems like most are accepted.

I am learning. Questions might seem dumb.

Thank you!

[boomer319]

Hello @gemenerik,

as suggested in your reply to this message I watched this talk by Francesco Paci from GWT.

He goes into detail on quantization using aquant at this time stamp.

This is the readme in the nntool directory of the gap_sdk git. Very informative.

There is no mention of input image pairs though.

Is there literature you would suggest on the topic in general?

Thanks!

[gemenerik]

What were the pictures displaying (e.g. samples/img_OUT_7_a.ppm)? What are the images supposed to display? Since these are used for quantization, they should represent typical inputs for the neural network?

Indeed, these can be images captured from the deck, or one could also use input images from an optical flow data set.

Following which rule would you group the images into pairs?

If, for example, you have three image pairs (where a and b form a pair) nntool expects them in the following order: 1_a 2_a 3_a 1_b 2_b 3_b. The tool will feed images a to input 1 and all images b to input 2, in order 1, 2, 3.

There is no mention of input image pairs though.

The talk is not about optical flow CNNs. Since we want to calculate the optical flow between two images, we feed two images to the network at a time. The image pairs are required by our application specifically, not quantization or nntool in general.

Is there literature you would suggest on the topic in general?

A good starting point is FlowNet

[boomer319]

Hello @gemenerik,

thanks for the reading suggestion! Much needed insight.

Did I get this right: Presuming I recorded a continuous set of 300 frames of AI deck footage: I sequentially divide that into 150 separate 2-frame-pairs: 1_a to 150_a and 1_b to 150_b Reasoning being: Each of the 150 a-b-pairs get processed individually by nntool for flow calculation.

[gemenerik]

Assuming you have a continuous set of 10 frames captured, let's call them 1, 2, 3 ... 10 in order of capture. You can feed them as follows: 1, 3, 5, 7, 9 (images a), 2, 4, 6, 8, 10 (images b). So that nntool calculates flow between from image 1 to 2 (1 and 2 form a pair), 3 to 4, etc.

But you can get more representative examples out of your captured set. You can also add: 2, 4, 6, 8 (images a), 3, 5, 7, 9 (images b). You can also reverse groups a and b, so nntool will calculate flow from image 2 to 1, 4 to 3, etc.