search

Xilinx / finn

Dataflow compiler for QNN inference on FPGAs
https://xilinx.github.io/finn
BSD 3-Clause "New" or "Revised" License
723 stars 230 forks source link

The model works with V0.9 but not with new layers in V0.10. #1055

Closed Arbiter-glitch closed 4 months ago

Arbiter-glitch commented 5 months ago

There is a considerable difference between the output and the expected output. Expected output is the output image received while passing the input image through the software model. I use the latest FINN branch. I trained the network outside the Finn docker. Then exported the qonnx format to Finn. Most of the nodes are rtl while some are hls. I have included the intermediate models and the build_dataflow_steps.py in the zip file as well as the output received through finn and the expected output. The notebook for running in Pynq is also included. I don't know whether I am doing anything wrong. Also, I took out the last mul node because it just divides by 255 to get the output between 0 and 1.
check.zip

[Update:] It works with old layers of v0.9 but not with the latest branch of FINN v0.10 with RTL layers. Since I want more throughput, RTL layers are better. But for some reason output is all white like, with some feature here and there. Hope this can be solved. Also I have targeted ZCU104 Ultrascale+ Evaluation Board

Expected Output

expected_finn

Actual Output Received

numpy_outfinn Originally posted by @Arbiter-glitch in https://github.com/Xilinx/finn/discussions/1052

Arbiter-glitch commented 5 months ago

[Update]:

To find out why the output is different in finn v0.10. I took Finn v0.9 and started to replace each of the old layers with new RTL layers, I checked at each stage. And it was only after replacing the old Thresholding hls layers with RTL layers, the output varied from the expected to give the white one again. So the problem might be with the thresholding rtl layers.

auphelia commented 4 months ago

See: Issue https://github.com/Xilinx/finn/issues/1060 and PR https://github.com/Xilinx/finn/pull/1077