sld-columbia / esp

Embedded Scalable Platforms: Heterogeneous SoC architecture and IP integration made easy
Other
338 stars 106 forks source link

Support for additional platforms #68

Open JuanEsco063 opened 4 years ago

JuanEsco063 commented 4 years ago

Currently we do not own any of the supported platforms but we are interested in adding ESP to our workflow. We do have other FPGA boards we would like to use.

I was wondering how easy would it be to add support for additional platforms. Namely the ZCU102 and Alveo U280.

Alternatively, I could try to generate the required files on my own but I do not see a tutorial or guide on that.

Thanks in advance.

paulmnt commented 4 years ago

Hello and thank you for writing! In general, supporting a different development board is not too complex, but there are three requirements:

  1. The FPGA (programming logic) must have direct access to DDR (satisfied for both ZCU102 and Alveo U280)
  2. The FPGA (programming logic) must have direct access to Ethernet (not satisfied)
  3. The FPGA (programming logic) must have direct access to a UART interface (satisfied for ZCU102, but not for Alveo U280)

We use a module from GRLIB to control the ESP instance via Ethernet, which means that the on-board Ethernet PHY should be connected directly to the FPGA fabric. The Alveo U280 does not have Ethernet, so it's not an option at the moment. The ZCU102, instead, does have it, but the on-board PHY is exposed to the PS-side Ethernet MAC, and not to the FPGA fabric. The PS-side Ethernet module does not support receiving debug messages; it can only be used as a normal peripheral.

Having said that, there is a more complicated, but feasible, solution for the ZCU102: it is possible to use one master port between the PS and the PL to control the ESP instance, thus replacing entirely the Ethernet debug module. This solution is more complex than just porting the constrain files, because it requires some design changes.

A design template for the ZCU102 board is something we would like to have, because the board is very popular. Would you be interested in contributing? This type of porting is not documented in a tutorial, because each FPGA board is slightly different and there is no single recipe to do it, but I can help with every step (feel free to email me if you are interested).

At a high level, these are the steps:

simplelins commented 3 years ago

@paulmnt hi, if we want to use the Alveo U200 to run with esp , could you give me some suggestions ? and more we will use vitis2020 to compiling project. how easy would it be to add support for u200 with vitis2020. Thanks in advance.

paulmnt commented 3 years ago

Hi @simplelins. Thank you for writing.

Bumping the IP definitions and the scripts for Vitis 2020 is not difficult. For the most part it just requires to change IP revision numbers.

Supporting the Alveo board requires a bit of work instead: the debug interface that currently relies on Ethernet in ESP, should be implemented using PCIe. That means designing an adapter for the PCIe IP to the AHB bus. In addition, a PCIe device driver must be implemented to control ESP. Also, I am not familiar with the Alveo boards, but I don’t see a UART interface in the specs. That means there is no direct console to interact with the cores in ESP and see programs output. A possibile workaround could be to have purchar() and getchar() customized: instead of driving the UART, those functions should issue a command over PCIe that reads or writes a memory region of the host computer. The host then should monitor the memory region and emulate the UART.

In practice, if you are familiar with PCIe, I think the porting can be done. Other than peripherals, the core components of ESP can be synthesized for any target FPGA.

paulmnt commented 3 years ago

Hi @simplelins: are you still attempting to create a template design for the Alveo board? Please let us know if you need additional information or help.

simplelins commented 3 years ago

Hi, I had done a try with hls4ml for the model resnet50. and that can not get a good tansformation. Now I use the finn do a try. thanks for your help!

GiuseppeDiGuglielmo commented 3 years ago

@simplelins,

I am active in both the ESP and hls4ml communities. On the hls4ml side, we will integrate the support for ResNet50 in the main branch of hls4ml soon. We have a working fork/branch in https://github.com/violatingcp/hls4ml/tree/sc20_test (in particular see if this example may help https://github.com/violatingcp/hls4ml/tree/sc20_test/example-models/resnet50-hls-full_300mus).

Did you try to reach the hls4ml community by opening a ticket on the tracking system https://github.com/fastmachinelearning/hls4ml/issues? We are happy to help you with your model translation.

If you decide to go with FINN, let us know if you will successfully integrate a FINN accelerator in ESP. This integration will be valuable for other peoples and later on, you will get feedback for additional improvements.

Best, Giuseppe

simplelins commented 3 years ago

@GiuseppeDiGuglielmo Hi, thank for your suggestion, I had a try using the branch "https://github.com/violatingcp/hls4ml/tree/sc20_test/example-models/resnet50-hls-full_300mus " at last year but some error occured , so I research for other tools to do a translation for resnet50. Up to now, the finn can translate the mobilenetv1 to the alveo card. and recently they will opensource the model of resnet50. so I will continue to follow with interest。