search

PeterOgden / ZCU104_VideoDemo

PYNQ demo as seen at FPL 2018
20 stars 7 forks source link

Accessing Optical Flow Frames #2

Open jamesthesken opened 5 years ago

jamesthesken commented 5 years ago

I know this was a demonstration and may be ignored/deprecated already but quick question.

Do you think it is possible to access the frames processed by the optical flow IP? Meaning that the frame could be interpreted as a Numpy array and processed further by OpenCV?

Please let me know. Thank you for making a well-done demo notebook for the ZCU104.

PeterOgden commented 5 years ago

It's definitely possible but you'll need to modify the IP and hardware design. The optical flow function I'm using outputs two streams of floating point numbers for the X and Y motion vectors. You could remove the format_output function in the optical_flow.cpp and return the arrays directly.

If you don't want to modify the hardware design you get some of the way there by using the current output formatting to return the raw x and y values, albeit only at 8-bit precision. All of the arithmetic is unsigned so you'll need to add a bias in the output format and then remove it in software.

All of the hardware arrays in PYNQ are numpy arrays so once you have the revised IP you can easily pass the results off to OpenCV