Xilinx / open-nic-dpdk

DPDK Drivers for AMD OpenNIC
Other
18 stars 8 forks source link

Bottleneck on the minimum size packet performance. #12

Closed function47 closed 7 months ago

function47 commented 7 months ago

Hi, Chris,

As mentioned in the OpenNIC FAQ, "This slower clock domain cannot handle the theoretical worst-case packet rate but reflects the fact that, for packets to and from the host over PCIe, the QDMA only runs at 250 MHz." I am wondering whether it is possible to improve the performance on worst-case packet rate for packets hitting the host? Or is it constrained by the hardware/QDMA limitations?

Thanks! Han

cneely-amd commented 7 months ago

Hi Han,

Hopefully I can explain this without too many details. The worst case for analysis, having to do with our 512 bit bus, is when continuously receiving back-to-back packets of length 65 bytes. In this case the repeating pattern on the data bus has first word of 64 bytes of data followed by the second word containing only one byte of valid data. Almost every other cycle is wasted.

If we assume a 300 MHz clock then we can handle properly receiving 150M packets/sec (for 100 Gb/sec).

If we slow down to a 250 MHz clock then we can only receive 125M packets/sec, with this alternating pattern.

The PCIe and QDMA IP use a 250MHz clock, limiting the worst case packet rate.

In practical situations packets will be much longer and this won't be a major concern.

Does this description help -- is this along the lines of what you were asking?

Thanks, --Chris

function47 commented 7 months ago

Thanks Chris. That makes a lot of sense.