Pakets counted twice, when using queues ?

[ghost]

Hi,

I recently use CPqD switch to examine meter & queue features. I think I stumbled over a strange issue, regarding the packet counter of flow, queue, and meters. Whenever I use a queue for forwarding packets, it seems that this packets gets counted twice. Lets assume I'm sending 100 packets, without queues .. everything is fine.. flow, meter and queue counters are equal. When I use queues on the same setup, the flow and meter counter seems to count the packets twice (means it says, that 200 packets are sent, but the queue counters show only 100 packets) I validated, that there are no packet drops and so on. I did some additional tests, sending data via iperf (udp) and measuring the bandwidth with counters of meters. When I send with 2Mbps, it shows 4Mbps (measured rate of meter), but only on that switches, in which the flows enqueues packets in queue. Does anyone can acknowledge this behaviour/bug ?

[balotr]

Hi @d-lobo

I have the same issue. In my switch I have 2 flow entiries. One for SET_QUEUE and METER action in table_id 0, one for OUTPUT action in table_id 1. I use same match structure in both flows. I think in my switch during table passes one packet is counted as two since I use two same matching structures.

To prove this I removed the flow entry in table_id 1 and run simulation again. This time queuing is not happened but meter counted packets as true (controlled from iperf sent datagrams)

Maybe the following idea is a must that when using queues one has to use two different matching structure such as in first table action dscp_remark and in second table match that dscp_remark. I will try this and write the result here

[balotr]

I tested the Queueing and Meter as follows:

h1-----(port1)s1(port2)-----h2

port1: Queue1(min rate 100), Queue0(min rate 10)

iperf flow to host1 UDP dst_port:5001

h1 IP: 10.0.0.1/8,gw:10.0.0.3 h2 IP:100.0.0.1/8,gw:100.0.0.3

Here is my flow table at the end of simulation:

mininet@mininet-vm:~$ dpctl tcp:localhost:6634 stats-flow

SENDING (xid=0xF0FF00F0):
stat_req{type="flow", flags="0x0", table="all", oport="any", ogrp="any", cookie=0x0", mask=0x0", match=oxm{all match}}

RECEIVED (xid=0xF0FF00F0):
stat_repl{type="flow", flags="0x0", stats=[{table="0", match="oxm{eth_type="0x800", ipv4_dst="10.0.0.1", ip_proto="17", udp_dst="5001"}", dur_s="657", dur_ns="370000000", prio="65533", idle_to="0", hard_to="0", cookie="0x1", pkt_cnt="35690", byte_cnt="53963280", insts=[apply{acts=[set_field{field:ip_dscp="63"}, queue{q="1"}]}, goto{table="1"}]},
{table="0", match="oxm{eth_type="0x800", ipv4_dst="10.0.0.3"}", dur_s="657", dur_ns="412000000", prio="1037", idle_to="0", hard_to="0", cookie="0x1", pkt_cnt="0", byte_cnt="0", insts=[apply{acts=[out{port="ctrl", mlen="65535"}]}]},
{table="0", match="oxm{eth_type="0x800", ipv4_dst="100.0.0.3"}", dur_s="657", dur_ns="388000000", prio="1037", idle_to="0", hard_to="0", cookie="0x2", pkt_cnt="0", byte_cnt="0", insts=[apply{acts=[out{port="ctrl", mlen="65535"}]}]},
{table="0", match="oxm{eth_type="0x800", ipv4_src="10.0.0.0", ipv4_src_mask="255.0.0.0", ipv4_dst="10.0.0.0", ipv4_dst_mask="255.0.0.0"}", dur_s="657", dur_ns="412000000", prio="36", idle_to="0", hard_to="0", cookie="0x1", pkt_cnt="0", byte_cnt="0", insts=[apply{acts=[out{port="normal"}]}]},
{table="0", match="oxm{eth_type="0x800", ipv4_src="100.0.0.0", ipv4_src_mask="255.0.0.0", ipv4_dst="100.0.0.0", ipv4_dst_mask="255.0.0.0"}", dur_s="657", dur_ns="388000000", prio="36", idle_to="0", hard_to="0", cookie="0x2", pkt_cnt="0", byte_cnt="0", insts=[apply{acts=[out{port="normal"}]}]},
{table="0", match="oxm{eth_type="0x800", ipv4_dst="100.0.0.1"}", dur_s="388", dur_ns="335000000", prio="35", idle_to="1800", hard_to="0", cookie="0x2", pkt_cnt="0", byte_cnt="0", insts=[apply{acts=[nw_dec, set_field{field:eth_src="3e:fe:f1:6f:25:fa"}, set_field{field:eth_dst="00:00:00:00:00:02"}, out{port="2"}]}]},
{table="0", match="oxm{eth_type="0x800", ipv4_dst="10.0.0.0", ipv4_dst_mask="255.0.0.0"}", dur_s="657", dur_ns="454000000", prio="2", idle_to="0", hard_to="0", cookie="0x1", pkt_cnt="0", byte_cnt="0", insts=[apply{acts=[out{port="ctrl", mlen="65535"}]}]},
{table="0", match="oxm{eth_type="0x800", ipv4_dst="100.0.0.0", ipv4_dst_mask="255.0.0.0"}", dur_s="657", dur_ns="412000000", prio="2", idle_to="0", hard_to="0", cookie="0x2", pkt_cnt="0", byte_cnt="0", insts=[apply{acts=[out{port="ctrl", mlen="65535"}]}]},
{table="0", match="oxm{eth_type="0x806"}", dur_s="669", dur_ns="836000000", prio="1", idle_to="0", hard_to="0", cookie="0x0", pkt_cnt="5", byte_cnt="300", insts=[apply{acts=[out{port="ctrl", mlen="65535"}]}]},
{table="0", match="oxm{eth_type="0x800"}", dur_s="669", dur_ns="835000000", prio="1", idle_to="0", hard_to="0", cookie="0x0", pkt_cnt="0", byte_cnt="0", insts=[apply{acts=[]}]},
{table="0", match="oxm{all match}", dur_s="669", dur_ns="835000000", prio="0", idle_to="0", hard_to="0", cookie="0x0", pkt_cnt="0", byte_cnt="0", insts=[goto{table="1"}]},
{table="1", match="oxm{eth_type="0x800", ip_dscp="63"}", dur_s="657", dur_ns="365000000", prio="55555", idle_to="0", hard_to="0", cookie="0x0", pkt_cnt="35690", byte_cnt="53963280", insts=[meter{meter="1"}, apply{acts=[out{port="1"}]}]},
{table="1", match="oxm{all match}", dur_s="669", dur_ns="855000000", prio="0", idle_to="0", hard_to="0", cookie="0x0", pkt_cnt="0", byte_cnt="0", insts=[apply{acts=[out{port="ctrl", mlen="65535"}]}]}]}

Here is my Meter stats:

mininet@mininet-vm:~$ dpctl tcp:localhost:6634 stats-meter

SENDING (xid=0xF0FF00F0):
stat_req{type="mstats", flags="0x0"{meter_id= ffffffff"}

RECEIVED (xid=0xF0FF00F0):
stat_repl{type="mstats", flags="0x0", stats=[{meter= 1"", flow_cnt="1", pkt_in_cnt="35690", byte_in_cnt="53963280", duration_sec="999", duration_nsec="812000000", bands=[{pkt_band_cnt="35690", byte_band_cnt="53963280"}]}]}

and here is my queue stats:

mininet@mininet-vm:~$ dpctl tcp:localhost:6634 stats-queue

SENDING (xid=0xF0FF00F0):
stat_req{type="queue", flags="0x0", port="any", q="all"}

RECEIVED (xid=0xF0FF00F0):
stat_repl{type="queue", flags="0x0", stats=[{port="1", q="0", tx_bytes="0", tx_pkt="0", tx_err="0"},
{port="1", q="1", tx_bytes="26981640", tx_pkt="17845", tx_err="0"}]}

Here is how I define Meters and Flow structure:

METER_1 = {"flags": "STATS", "meter_id": "1","bands": [{"prec_level": "1", "rate": "500000", "type": "EXPERIMENTER"}]}
RULE_1 = {"priority": "65533","match": {"dl_type":"IPv4", 
                                        "nw_dst": "10.0.0.1", 
                                        "nw_proto": "UDP", 
                                        "tp_dst": "5001"}, 
                            "actions": {"queue": "1", "mark": "63"}}

FLOW_1 = { "dpid": 1, "table_id": 1, "priority": 55555, "match":{ "dl_type": 0x800, 
                                                                 "ip_dscp": 63},
                                                        "actions":[ { "type":"OUTPUT", "port": 1 },
                                                                    {"type": "METER", "meter_id": 1}]
          }

And finally here is my Iperf output at the end of simulation:

[ 13] 197.0-198.0 sec   128 KBytes  1.05 Mbits/sec
[ 13] 198.0-199.0 sec   129 KBytes  1.06 Mbits/sec
[ 13] 199.0-200.0 sec   128 KBytes  1.05 Mbits/sec
[ 13]  0.0-200.0 sec  25.0 MBytes  1.05 Mbits/sec
[ 13] Sent 17835 datagrams
[ 13] WARNING: did not receive ack of last datagram after 10 tries.
root@mininet-vm:~# 

It is obvious that Iperf sends around 17830 datagrams and queue stats are true. However, meter and flow stats are counted as 2 times. I think between table transition, flows are counted again in statistics. In order to avoid such action, I used different match stucture in table 1, and mark the flow in table 0. However, it did not worked at all.

[renzoarreaza]

I'm having a similar issue. What I see is that when packets are send to a queue, a duplicate goes back into the switch through the output interface. I'm therefore receiving packets with an ip_src and in_port that in reality isn't possible.

This also explains what you guys are seeing, as you aren't matching on in_port and thus the duplicate packet will match the same entries again.