search

corundum / corundum

Open source FPGA-based NIC and platform for in-network compute
https://corundum.io/
Other
1.63k stars 410 forks source link

U200 packet receiving performance test #38

Open beer-belly opened 3 years ago

beer-belly commented 3 years ago

Hello, doctor about U200 board performance test, I want to ask you some questions.

test environment: We use Ixia device to do the packet test for U200 board. Ixia sets pseudo-random IP address and port number to ensure that packets can be evenly distributed to each queue, and the packet size is 9600 bytes The main parameters of U200 board are as follows: Rx MTU=16384, rx fifo depth=131072, number Rx queue=4096, rx pkt table size=16

Test results: Through PF_ Ring count instance packet receiving test, only supports 23Gbps no packet loss reception Set the count in engine.v . The counter is set after the queue response is completed and the data is written to ram, it can reach 37Gbps and 40Gbps respectively by reading the register In Xilinx's CMAC IP core, RX data is returned to TX for direct forwarding, and the loopback rate can reach 96Gbps

So what other configurations do I need to do to achieve the high-speed rate of 95.5Gbps mentioned in your paper

the result of 35Gbps flow image image

the result of 40Gbps flow image image

the result of 50Gbps flow image image

alexforencich commented 3 years ago

Can you provide any more information about the host system?

beer-belly commented 3 years ago

Thank you very much for your answer. CPU:Intel(R) Xeon(R) CPU E5-2609 v3 @ 1.90GHz physical id is : 2 processor is : 12 mem: 128G DDR3 图片1

I added counters in rx_engine.v image image

alexforencich commented 3 years ago

First, your PCIe link is only gen 3 x8, so there is no way you're going to break 50-60 Gbps. I would say hitting 40 Gbps reliably is quite reasonable. If you want to do any better, you'll need to figure out how to get the card into a true x16 slot, possibly you need to pull the GPU and put the FPGA in that slot, and leave the other x16 slot empty. Also, those are very old CPUs. I have had issues with some of the older Xeon CPUs not going over 50 Gbps, even with a x16 link. I have not been able to figure out why, but it's not a Corundum problem as the Mellanox NICs I tested had the same problem. Only thing I can think of is that it's some limitation of the CPU uncore. Try a commercial NIC and see what kind of performance you get. Or try with a more recent CPU.