Query: (1) why throughput higher than theoretical, (2) why lower throughput with 128B payload.

[108anup]

I am not sure what is the best place to ask this question, hence asking here as an issue. Please let me know if some other place is preferred over this.

Q1: I get cases where observed throughput is higher than theoretical maximum, e.g., 97 Gbps throughput with 1472B payload where the theoretical is around 95 Gbps. 2 Gbps is large enough difference that I can't explain using a few cycle measurement mismatch. I noticed the recently changed juptyer notebook also shows this but the mismatch is only 0.0005 Gbps or so. What might be the reasons for higher than theoretical throughput measurement. If we under measure the time to receive packets by even 100 cycles, that may explain 0.0005 Gbps mismatch but not 2 Gbps mismatch.

Q2. I observe relatively lower throughput with 128B payload. Why might that be? Is it to do with bytes transmitted per flit? If so, 64B payload then should also have similar issue (55 bytes per flit). But we see that only 128B payload has ~5 Gbps gap to theoretical maximum, while other payload sizes don't have much gap.

Calculation: Flit is AXI transfer data width (64B). For 128B payload, frame size = 128 + UDP (8), IP (20), Ethernet(14) and FCS (4) = 174B, requires 3 flits (3 * 64B >= 174B). Or 174/3 bytes sent per flit = 58. Similarly for 64B payload, bytes per flit is 55. For other payload sizes bytes per flit is >= 60.

[mariodruiz]

Hi,

1. Under what conditions are you getting this large mismatch? The CMAC should limit the maximum tx/rx bandwidth. However, if the burst is short, the FIFOs in the datapath can absorb this difference.
2. There's an overhead in the HLS IP which is impacts significantly segments in the range of 65 to 160 bytes.

In the throughput notebooks the overhead is shown:

# overhead is UDP (8), IP (20), Ethernet(14), FCS (4), IFG (12), preamble (7), start frame delimiter (1)
overhead = 8 + 20 + 14 + 4 + 12 + 7 + 1

[108anup]

1. I am using the provided notebook. I change MAC/IP address configuration and kernel configuration. I don't make any changes to the FPGA code, i.e., use vanilla xclbin. The TX/RX measurement is done on a single FPGA instead of 2 different FPGAs. The benchmarking FPGA sends packets to a device-under-test (DUT, running open NIC). The DUT forwards packets back to the benchmarking FPGA. The benchmarking FPGA has 2 kernels started (one in PRODUCER and other in CONSUMER mode). The PRODUCER kernel continuously sends packets (not in burst). 1.1. For 64B UDP payload (174B frame or packet), the TX throughput itself is around 1.163 Gbps above theoretical. This difference is at the frame level. At the application (payload), the difference is 0.67 Gbps above theoretical. 1.3 I.e., for 64B payload, throughput reported by notebook = 49.907 Gbps, frame_level_throughput = 85.777 Gbps. Theoretical maximum throughput = 49.231 Gbps at application level and = 84.651 Gbps at frame level. 1.2 (frame_level_throughput = application_level_throughput * (payload + 46) / payload).
2. 64 byte payload (frame size = 110B) should also be impacted by this overhead right. But only 128 byte payload (frame size = 174B) is impacted. From the notebook in upstream code, the throughput difference to theoretical max is 4.6 Gbps for 128B payload (174B segment or frame) while it is 0 for 64B payload (110B segment or frame). Segment or frame refer to bits transferred in one AXI stream transaction. 3.1. From what I understand, based on the reasoning you give, the impact should be witnessed by 64B payload not 128B payload as 64B payload creates segments of size 110B which lie in the range 65 to 160 bytes.

[108anup]

To double check, I can run the vanilla un-modified VNx notebook to see if the difference is still there. Since the difference is in TX throughput itself, I would expect the same difference to still be there as TX should not be affected by DUT much. Increase in RX throughput could be maybe from packet duplication (though I have no reason to believe that packets would be duplicated).

[mariodruiz]

1.1. For 64B UDP payload (174B frame or packet)

How do you get the 174B in here?

Each of the individual IP that compose the network layer needs one or two clock cycles extra to process each packet/segment. I suppose that for 128-Byte the extra cycles stack impacting the throughput. This is something I haven't profiled, because for bulk data transfer you will not use small packet size.

The low performance for small packet is known. Given the current design, this is unavoidable.

Mario

[108anup]

Sorry typo there. For 64B payload (64+46=110B frame)

[108anup]

I understand that performance will be low for small packets. The thing I don't understand is why performance for 64B payload is better than performance for 128B payload. I get better throughput with 64B payloads.

Throughput I measure for payload level using provided notebook is 49.907 Gbps for 64B and 60.21 Gbps for 128B. If we translate these to frame level, using frame_level_throughput = payload_level_throughput * (payload size + header size) / (payload size), we get throughput as 85.77 Gbps for 64B payload (110B frame) and 81.84 Gbps for 128B payload, i.e., packet level throughput is higher for 64B payloads.

Even in the provided notebook, the throughput with 128B payloads is 5 Gbps lower than theoretical while for 64B payloads, it is close to theoretical, i.e., the efficiency for 64B payloads (smaller payload) is better than that with 128B payloads (larger payload).

[108anup]

Also any thoughts on why TX throughput might be higher than theoretical?

[mariodruiz]

This is how I compute these throughputs

udp = 8
ip = 20
eth = 14
fcs = 4
ifg = 12
pp_amble = 8

def thr(payload_size: int):
    total_bytes = payload_size + udp + ip + eth + fcs + ifg + pp_amble
    payload_thr = payload_size / total_bytes
    frame_thr = (payload_size + ip + udp + eth) / total_bytes
    return payload_thr * 100 , frame_thr * 100.0

So, thr(64) = (49.23076923076923, 81.53846153846153) and thr(128) = (65.97938144329896, 87.62886597938144)

I think your theoretical equation does not look right, as the payload (segment) size increases, the overhead decreases (the deficiency increases).

Correct me if I am missing something.

[mariodruiz]

Someone already asked about this here, I suppose your theoretical throughput is what this person calls naked cmac, still does not mach the numbers I showed above. But, I believe the Python snippet is the correct way to compute this.

[108anup]

My defs are same except: frame_thr = (payload_size + ip + udp + eth + *fcs*) / total_bytes Frame is what goes into tdata of the AXIS interfaces, i.e., payload + 46 bytes. This is also why tkeep in LATENCY kernel is 18 = 64 - 46, i.e., 18B payload gives frame of size 1 flit (TDATA WIDTH).

1. For the small packets, the issue is not in theoretical computation. The issue is that the measured frame throughput for 128B payload is smaller than 64B payload (in my setup). Even in the upstream notebook, the measured payload throughput for 128B payload is further away from theoretical (4 Gbps away, 65 Gbps theoretical, 61 Gbps measured) than 64B payload (0 Gbps difference, measured and theoretical = 49.231 Gbps). Even here, the measured L2 bandwidth is higher for 110B packet (95.45) compared to 174B (93.84).

2. For theoretical calculations (at least for theoretical throughput at payload level), I use exact same calculation as yours. I am seeing higher measured throughput than theoretical throughput at payload level of around 0.6 to 1.3 Gbps.

Payload | Theoretical Gbps at payload level | Measured Gbps at payload level | Difference -- | -- | -- | -- 64 | 49.23076923 | 49.90756385 | -0.6767946191 128 | 65.97938144 | 60.21115183 | 5.768229612 192 | 74.41860465 | 75.26383693 | -0.8452322772 256 | 79.50310559 | 80.59648509 | -1.093379504 320 | 82.9015544 | 84.04169256 | -1.140138154 384 | 85.33333333 | 86.50693924 | -1.173605905 448 | 87.15953307 | 88.35825842 | -1.198725344 512 | 88.58131488 | 89.79962376 | -1.218308883 576 | 89.71962617 | 90.95361477 | -1.233988597 640 | 90.65155807 | 91.89833917 | -1.246781098 704 | 91.42857143 | 92.68604922 | -1.257477787 768 | 92.08633094 | 93.35285902 | -1.266528087 832 | 92.65033408 | 93.92464052 | -1.274306441 896 | 93.13929314 | 94.42033049 | -1.281037355 960 | 93.56725146 | 94.85417209 | -1.286920625 1024 | 93.94495413 | 95.23706712 | -1.292112994 1088 | 94.28076256 | 95.57749555 | -1.296732985 1152 | 94.58128079 | 95.88214486 | -1.300864075 1216 | 94.85179407 | 96.15638323 | -1.304589155 1280 | 95.09658247 | 96.40453013 | -1.307947665 1344 | 95.31914894 | 96.630162 | -1.311013068 1408 | 95.52238806 | 96.83620318 | -1.313815124 1472 | 95.70871261 | 97.02508574 | -1.316373129

In above table the measured at payload level is directly reported by the notebook, i.e., from ol_w0_tg.compute_app_throughput('tx') . The theoretical is taken from what you described here, i.e., payload_thr = 100 * payload_size / (payload_size + udp + ip + eth + fcs + ifg + pp_amble).

In this table, there are 2 weird things,

1. The behavior for 128B payload is different compared to smaller and larger payloads (difference = +5.76Gbps vs -0.67 Gbps (64B) and -0.84 Gbps (192B).
2. There is significantly larger than theoretical throughput (e.g., 97.02 Gbps measured vs 95.708 Gbps theoretical for 1472B payload).

[mariodruiz]

How many packets are you sending for each payload size?

1. 128-Byte is always going to give the worst efficiency. Without redesigning the whole network layer, this cannot be solved.
2. There's a small chance that the CMAC is overclocked. This only could happen when connected to another Alveo card. You can try to run the same experiments when VNx is connected to a different network equipment (no Alveo).

[108anup]

For the above table, I sent 1 billion packets (basically the PRODUCER is set as in the notebook). Results are similar for 1 million packets as well.

1. Could you please elaborate why 128B is worst than 64B payload.
2. Hmm.., so the QSFP and wire support more than 100Gbps? In my setup, the FPGAs are connected to a 100G switch.

[mariodruiz]

1. I mentioned above, each IP need at least one more cycle to process each segment. For 128-Byte, these stack creating the highest overhead.
2. The QSFP28 has four line that can work up to 28 Gbps each.