hpcn-uam / Limago

Limago: an FPGA-based Open-source 100 GbE TCP/IP Stack
BSD 3-Clause "New" or "Revised" License
122 stars 46 forks source link

cuckoo_cam is a dcp built on xcvu9p-flga2104-2L-e #3

Open luyong6 opened 4 years ago

luyong6 commented 4 years ago

What is cuckoo_cam? It is a checkpoint built on xcvu9p-flga2104-2L-e now. Is it possible to build it to other cards? If it can also be generated by tcl or source RTLs that makes things more easier. Thanks.

mariodruiz commented 4 years ago

CuckooCAM is a Content Addressable Memory based on the cuckoo hashing and other few improvements. It is used to convert an incoming three tuples to a unique ID. Maybe Figure 1 here gives a better understanding. What board are you interested in? I could provide a dcp for that particular FPGA.

luyong6 commented 4 years ago

Oh, xcvu37p-fsvh2892-2-e I just realized that this dcp still passed the synthesis even the fpga_part has been changed to vu37p.

Maybe because it is just a synthesized netlist? Please hold on and really appreciated.

mariodruiz commented 4 years ago

The current dcp should implement in any VirtexUltrascale+ except on VirtexUltrascale+ HBM. Yes, it is a synthesized netlist for a family of devices.

The plan is to release the code, but there are some improvements to be done and more experiments.

luyong6 commented 4 years ago

Thanks Mario

luyong6 commented 4 years ago

Is there any plan to relase the code? I realized although the bitstream was written, there were some bad timing paths around cuckoo_cam. I don't know whether this is the cause that I still didn't get rx_aligned.

image

mariodruiz commented 4 years ago

Hi @luyong6, There is no plan to release the code at the moment. This should not cause problems with the alignment, but I can be related.

Where are you connecting the FPGA to test link?

luyong6 commented 4 years ago

Oh,

According to UG203 (Version 3.1, June24 2020) image

I modified the FSM sequence in submodules/cmac/src/cmac_sync/cmac_0_axi4_lite_user_if.v

189                 STATE_INIT_RX_ALIGNED    : begin
190                      rx_busy_led_r           <= 1'b1;
191                      init_rx_aligned         <= 1'b1;
192
193                     case (rd_wr_cntr)
194                         'd0     : begin
195                             $display( "           AXI4 Lite Write Started to Config the Core CTL_* Ports ..." );
196                             axi_wr_data             <= 32'h00000001;           //// ctl_rx_enable
197                             axi_wr_addr             <= ADDR_CONFIG_RX_REG1 + SLAVE_CMAC_BASEADDR;    //// CONFIGURATION_RX_REG1
198                             axi_wr_addr_valid       <= 1'b1;
199                             axi_wr_data_valid       <= 1'b1;
200                             axi_wr_strobe           <= 4'hF;
201                             axi_rd_req              <= 1'b0;
202                             axi_wr_req              <= 1'b1;
203                         end
204                         'd1     : begin
205                             axi_wr_data             <= 32'h00000010;          //// ctl_tx_send_rfi, according to pg203, Version June 24, 2020
206 //                            if (ULTRASCALE_PLUS==0)
207 //                                axi_wr_data             <= 32'h00000010;          //// ctl_tx_send_rfi
208 //                            else
209 //                                axi_wr_data             <= 32'h00000018;          //// ctl_tx_send_lfi, ctl_tx_send_rfi
210                             axi_wr_addr             <= ADDR_CONFIG_TX_REG1 + SLAVE_CMAC_BASEADDR;   //// CONFIGURATION_TX_REG1
211                             axi_wr_addr_valid       <= 1'b1;
212                             axi_wr_data_valid       <= 1'b1;
213                             axi_wr_strobe           <= 4'hF;
214                             axi_rd_req              <= 1'b0;
215                             axi_wr_req              <= 1'b1;
216                         end

However, the rx_aligned seemed not get assigned. So I think this FSM was stuck in waiting for it. I was using Vivado 2020.1, on a device with vu37p device.

The xdma driver was successfully loaded. Some information:

 sudo ./bin/reg_rw  /dev/xdma0_user 0x24
...
Read 32-bit value at address 0x00000024 (0x7f275c71c024): 0x00000301   CMAC subsystem version is correct: 3.1

sudo ./bin/reg_rw  /dev/xdma0_user 0xC
...
Read 32-bit value at address 0x0000000c (0x7f430b82200c): 0x00000010  (This means ctl_tx_send_rfi)
sudo ./bin/reg_rw  /dev/xdma0_user 0x14
Read 32-bit value at address 0x00000014 (0x7fde06d05014): 0x00000001 (This means ctl_rx_enable)
sudo ./bin/reg_rw  /dev/xdma0_user 0x204
Read 32-bit value at address 0x00000204 (0x7f651d54a204): 0x000000c0 (stat_rx_local_fault, stat_rx_internal_local_fault)

The same card, if I don't use AXI4Lite interface to configure the CMAC but directly assign ctl_tx_enable/ctl_rx_enable, with a 100G cable connecting to a Mellanox CX4 card, it had rx_aligned, and CX4 card also said "Link UP".

mariodruiz commented 4 years ago

@luyong6 the bring up sequence change slightly. But, I am using the same code with Vivado 2020.1 and it is working. The fact that you can link up writing the registers from the host means that something is not working properly in the interconnection between the cmac_synq and cmac. How is lbus_tx_rx_restart_in connected? My block design is exactly this one

Maybe you can generate the example design for the CMAC with AXI4-Lite and check the code.

edubeZetec commented 4 years ago

Hi, I want to port the design to a kintex device any chance you release de code ? or provide a .dcp for the kintex ?

Thx

mariodruiz commented 4 years ago

Hi,

We are not planning to release the code in the short-term.

Kintex 7 or Kintex US(+), if the latter it should work. If the former, you should check ETH implementation, they have support for series 7. https://github.com/fpgasystems/fpga-network-stack

At this point I will be difficult to provide the dcp. I will try though.

Mario

luckymonkey227 commented 4 years ago

Hello. I can't add the cuckoo_cam.dcp into the project vcu118_fns_single_toe_iperf. The error message is: '../cuckooCam/cuckoo_cam.dcp' is not a valid design checkpoint. Could you upload the correct file again? Thanks.

mariodruiz commented 4 years ago

does the cuckoo_cam.dcp file exist? what it is its size? The file is staged with git lfs, and maybe you did not pull it from the repo.