Data Mismatch in Default FINN FPGA Flow Tutorial Using MNIST Numerals Example

[prospersu]

Prerequisites

• [Y] Test that the bug appears on the current version of the dev-branch. (Make sure to include the commit hash of the commit you checked out).
• [Y] Check that the issue hasn't already been reported, by checking the currently open issues.
• [Y] If there are steps to reproduce the problem, make sure to write them down below.
• [Y] If relevant, please include the ONNX files, which were created directly before and/or after the bug.

Quick Summary

When running the FINN FPGA flow using the MNIST Numerals example, the output results do not match the expected values. Despite compiling the code successfully, the simulation shows data mismatches and zero validated data points.

Details

Steps to Reproduce

1. Clone the FINN repository.
2. Checkout the dev branch, with commit hash.
3. Start the Docker container with the command.
4. Run transformation on the ONNX file or run the dataflow builder inside the tutorial folder.
5. Compile the FPGA flow using the MNIST Numerals example.
6. Observe the mismatch in output during the simulation.

Expected Behavior

The simulation should produce output data that matches the expected values, with zero error counts and validated data points.

Actual Behavior

The simulation completes with errors:

• Total error count: 1
• Mismatch between the expected data and the output data in the testbench (lines 151-162 of the code).
• The validation process reports zero validated data points.
• The examples are shown below

Possible Fix

Potential issues could be related to data misalignment between the input samples and expected outputs or timing issues in the testbench synchronization. Verifying the pointers (wr_ptr, rd_ptr) and ensuring data transfer synchronization in the testbench might resolve the problem.

Additional Context

• The error occurs even without any code modification from the original FINN example.
• Relevant ONNX files and simulation waveforms can be provided upon request.

[auphelia]

Hi @prospersu ,

Thanks for creating the issue, could you check if this PR solves your issue: https://github.com/Xilinx/finn/pull/1212

The folding configuration wasn't updated with the latest FINN version and so the folding wasn't applied. That might have caused a mismatch between expected folding from the test bench and the actual FINN IP.