search

cpignata / powerbench

Benchmarking Methodology for Power
0 stars 1 forks source link

Do we need to consider packet loss case? #10

Open billwuqin opened 6 months ago

billwuqin commented 6 months ago
  1. Section 2.2 of draft-cprigf-bmwg-pwoerbench said: “ A tolerance on packet loss and/or forwarding error must be specified somehow. That tolerance could be zero for some benchmark problems, and non-zero for others.

” It looks ETSI/ITU-T/ATIS more focus on zero packet loss case, e.g., See section 6.2 of ETSI-ES-203-136 " The 100 % data rate is the maximum NDR (Non Drop Rate) supported by the equipment: • The capacity of the equipment shall be obtained from traffic generator, unit is Gbps, the sum of bandwidth of all the ports calculated cannot be used in EEER calculation. " Also See Section 6.3 “Measurement Procedure” of ETSI-ES-203-136: " Step 4: Qualification of Maximum Traffic load and Capacity Measurement With the equipment configured as stated above, the equipment shall be operated for at least 15 minutes for initialization until equipment run stable, after that, configure the equipment according to clause 6.2 and make the data rate of all ports reach to the rate with no packet loss, measure the capacity of all in egress direction and record them, this capacity is the maximum capacity(Tmax) for each port. If some ports reach to line rate but lose packet, adjust the capacity of those ports until no packet loss, record the capacity of each egress port, calculate the total capacity(Ti) of all ports. " It looks in most case we require no packet loss, e.g., measure throughput, do we have case for tolerating packet loss?

romain-jacob commented 6 months ago

I am not sure about this, to be honest. Starting with the "no packet loss" case might be the simpler thing to do.

On the other hand, I don't really like the idea that each device would set it's own Tmax. Say that device A does a better job at forwarding than device B for the same port spec (say, 100G). The benchmark should/could show this by measuring higher packet loss rates for B than A... But TBH I don't think losses are the focus of this benchmark.

I think going for "no loss allowed" scenarios is the sensible thing, at least to start with.

cpignata commented 6 months ago

Im also of the opinion that no packet loss is easier and better. We needn't replicate ETSI and only bring things that are realistic.

giuseppefioccola commented 6 months ago

I also agree to start with no packet loss.