Xilinx / finn

Dataflow compiler for QNN inference on FPGAs
https://xilinx.github.io/finn
BSD 3-Clause "New" or "Revised" License
708 stars 225 forks source link

Custom build flow with stitched IP doesn't have TLAST #836

Closed ash8327 closed 1 year ago

ash8327 commented 1 year ago

Current Version commit b3bdff1

I am trying to use the stitched IP approach for a non-PYNQ FPGA board. I can successfully output the stitched IP design but when IP-XACT is generated it doesn't have a TLAST streaming connection for use with a DMA. Is there an example of how the custom design approach is done?

Steps to Reproduce

Follow Cybersecurity notebook but substitute another FPGA part to create stitched IP. Instantiate the IP in another design as explained https://github.com/Xilinx/finn/tree/main/tutorials/fpga_flow under Instantiation in Mission Design

Expected behavior

Us DMA IP to connect streaming inputs and outputs (S2MM and MM2S)

Actual behavior

TLAST not connected which will result in DMA not functioning correctly.

Possible fix

Add the TLAST connection as an option during IP creation

Additional context

No examples for non-PYNQ devices but this would make FINN easier to use across FPGA parts.

image

image

auphelia commented 1 year ago

Hi @ash8327 ,

You could use the InsertTLastMarker transformation on your network.

For integrating your stitched IP into a different FPGA board, please have a look at this discussions question: https://github.com/Xilinx/finn/discussions/387 The tutorial is published here: https://github.com/Xilinx/finn/tree/main/tutorials/fpga_flow

ash8327 commented 1 year ago

Thanks @auphelia ,

I managed to insert a TLAST using the AXI Stream Subset Converter. I also got a simple driver written with Vitis that runs this example on a MINIZed without PYNQ. I am trying to understand the differences between IODMA and the what the InsertTLASTMarker does.

I have focused mainly on the implementation in hardware currently and haven't spent much time on the transformation side. I looked at the documentation but I am not seeing how to use the InsertTLastMarker for the Cyber security example, could you give me some guidance on when I would use that transformation?

Other than that guidance, this can be closed.

auphelia commented 1 year ago

Hi @ash8327 ,

Great to hear that you were able to run your example on MINIZed without PYNQ! 🙂

In FINN, the DMAs act like a shell surrounding the hardware design. They are set up in a way that the amount of data they receive or pass on is determined, that is why the FINN designs don't use a TLAST marker anymore. When calling the ZynqBuild or VitisBuild transformation, which will be called under the hood when the step_synthesize_bitfile is executed, they get automatically inserted. For your example, you cannot use the standard flow, because that doesn't contain any TLAST marker insertion. You could either let the cybersecurity example generate the stitched IP for you and then do the TLAST marker insertion in the GUI or you could write your own custom_step like it is done in some finn-examples that also require extra transformations that are not included in the standard builder steps. Custom steps can be integrated by passing a list of steps to the builder tool, like it is done for MobileNet-v1 in finn-examples.

JPPalacios commented 1 month ago

Hi @auphelia @ash8327,

Wondering if this is still the case for the latest version of FINN? I want to stream data into StreamingDataflowParitition_1 but keep odma0 for PS post-processing.

So far, I have my MIPI + FINN pipelines working fine via DMAs but wanted to streamline the input side with HLS Letterboxing.

Many thanks!