search

aws / aws-fpga

Official repository of the AWS EC2 FPGA Hardware and Software Development Kit
Other
1.49k stars 513 forks source link

LUT utilization target #545

Closed useragdp1 closed 2 years ago

useragdp1 commented 2 years ago

My design is consumes ~55% LUTs. The RAM and DSP utilization is pretty low < 10%. When I try add more logic it runs into congestion issues. So, I am trying to reduce some bus widths. However, I would like to know if there is a good rule of thumb on what LUT utilization can be target without hitting congestion issues. I am hoping you guys have such data from various customer datapoints. Thanks

AWScsaralay commented 2 years ago

Hello,

While the 55% LUT utilization may not be the primary reason for congestion, the tool may still struggle to close timing if there are multiple SLR crossings, datapaths with greater than 10 levels of logic and heavy usage of global asynchronous resets. Please try addressing these issues, and also add multiple pipeline stages to spread out the logic in entire SLRs.

Also, please try floorplanning the design based on SLRs to be used. An example to constrain a module to a specific SLR is as shown below: https://github.com/aws/aws-fpga/blob/master/hdk/cl/examples/cl_dram_dma/build/constraints/cl_pnr_user.xdc#L3

Additional resources to help timing closure: https://github.com/aws/aws-fpga/blob/master/hdk/docs/AWS_Shell_Interface_Specification.md#implementation-tips https://www.xilinx.com/content/dam/xilinx/support/documentation/sw_manuals/xilinx2021_2/ug1292-ultrafast-timing-closure-quick-reference.pdf https://www.xilinx.com/content/dam/xilinx/support/documentation/sw_manuals/xilinx2021_2/ug949-vivado-design-methodology.pdf#nameddest=TimingClosure

Please feel free to contact us if you have any questions.

Thanks! Chakra

useragdp1 commented 2 years ago

Hi Chakra, I am able to resolve congestion using some of techniques you pointed to. But, we want to push more logic into the FPGA. For example, it seems obvious 80% utilization would be impossible for us to route (I have tried). So, the number is somewhere between 55-80% which is a large range to run experiments with. In order to reduce our development time, I am looking for some guidance on how much more LUTs can we practically push into this FPGA. Some datapoints would be very helpful. Thanks

AWScsaralay commented 2 years ago

Unfortunately, since each design is different in terms of pipeline stages, floorplanning, place and route, etc, it is difficult to pin-point a "safe" utilization number. We have seen designs with > 70% LUT utilization close timing at 250MHz, but with very efficient/optimized RTL datapaths and floorplan constraints.

Alternately, you could try running design at slower clock speeds (if that is an acceptable solution). Also, if your appllication does not rely on XDMA engine then I would recommend you to try our Small Shell which is 30% smaller in size and offers more resources to the Customer Logic: https://github.com/aws/aws-fpga/tree/small_shell#small-shell

Please let me know if you would like any additional details.

Thanks! Chakra

useragdp1 commented 2 years ago

Hi Chakra, I would like to use the small_shell since we do not use XDMA. Do I really need to rollback the Vivado from 2021.1 to 2020.2? Also, would it require change in SW driver? We have some working version with the current shell. Thanks Abhay

AWScsaralay commented 2 years ago

Hello Abhay,

Small Shell is supported on vivado tool versions 2020.2 and above. Therefore you don't need to roll back to 2020.2 if you are currently using 2021.1.

Small Shell has same interfaces between Shell and CL as provided by the full sized F1.X.1.4 Shell. Please note that the floorplan location of PCIM and PCIS interfaces have changed in Small Shell. Please refer to the guidelines here https://github.com/aws/aws-fpga/blob/small_shell/hdk/docs/AWS_Shell_Interface_Specification.md#multi-slr-fpga for optimal floorplanning of your CL. Since the PF/BAR ranges did not change, SW driver that worked with full sized F1.X.1.4 Shell should also work with Small Shell.

Please let us know if you see any issues or have any questions.

Thanks! Chakra

useragdp1 commented 2 years ago

Thanks Chakra

useragdp1 commented 2 years ago

I bypasses the Vivado version check in hdk_setup for small_shell. But ran into these issues while synthesising.

WARNING: [IP_Flow 19-2162] IP 'vio_0' is locked:

INFO: [Synth 8-6157] synthesizing module 'sh_ddr' [/data//awssmall/hdk/common/shell_v04182104/design/sh_ddr/synth/sh_ddr.sv:32] Parameter DDR_A_PRESENT bound to: 1 - type: integer Parameter DDR_B_PRESENT bound to: 1 - type: integer Parameter DDR_D_PRESENT bound to: 0 - type: integer ERROR: [Synth 8-5809] Error generated from encrypted envelope. [/data//awssmall/hdk/common/shell_v04182104/design/sh_ddr/synth/sh_ddr.sv:610] ERROR: [Synth 8-5809] Error generated from encrypted envelope.

Finished RTL Elaboration : Time (s): cpu = 00:00:19 ; elapsed = 00:00:24 . Memory (MB): peak = 3275.777 ; gain = 453.438 ; free physical = 13192 ; free virtual = 26431

RTL Elaboration failed

czfpga commented 2 years ago

Hi abhaypelago,

Thank you for reaching out. Looks like we might need to add support for version 2021.1. I did bypass the vivado version check and build the example of CL_DRAM_DMA in 2021.1 and saw no error during RTL elaboration. Is this from building a CL examples or your own CL design? Thanks.

useragdp1 commented 2 years ago

thanks for the hint. I should have copied the build files from cl_dram_dma. I am able to make progress. When can vivado version 2021.1 be support be added? I am proceeding by skipping the check and try the image next week.

useragdp1 commented 2 years ago

The small_shell is unable to make under cl_dram_dma/verif/scripts ERROR: [VRFC 10-2063] Module not found while processing module instance [/data/awssmall/aws-fpga/hdk/common/shell_v04182104/design/ip/axi_register_slice_axuser/sim/axi_register_slice_axuser.v:318] Note that I had to remove xsc cmd from Makefile since --additional_options is deprecated in newer vivado version

czfpga commented 2 years ago

Hi abhaypelago,

Thank you for sharing the updates. These are very helpful data points that I'll bring back to the development team. I believe at this moment, the best option is to use Vivado 2020.2 for your project since that's the version officially supported and has been validated. The development team is looking at adding supports for newer tool versions. There is no ETA for that yet.

Thanks again,

Chen

czfpga commented 2 years ago

Hi abhaypelago,

I'll close this issue. Please feel free to open a new one if you have any other questions. Thanks.

Chen