search

pinggit / dpdk-contrail-book

contrail dpdk day one book
9 stars 3 forks source link

lcore mapping issue (single queue) #27

Closed pinggit closed 3 years ago

pinggit commented 3 years ago

something is not right here... (or maybe I missed sth).

here I make PROX a single queue interface. and dpdkinfo -c shows lcore1 connects it. but per dpdkvifstats.py traffic landed core0 still. I confirmed the script is right ( only vif --get 3 --core 10 shows the traffic). can this be explained?

[root@a7s3 ~]# contrail-tools dpdkvifstats.py -v 3 -c 2
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|Core 1  | TX pps: 0         | RX pps: 85221     | TX bps: 0         | RX bps: 10567445  | TX error: 0         | RX error 0         | TX port error: 0         | RX queue error 0         |
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|Core 2  | TX pps: 85262     | RX pps: 1         | TX bps: 10572447  | RX bps: 42        | TX error: 0         | RX error 0         | TX port error: 0         | RX queue error 0         |
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|Total   | TX pps: 85262     | RX pps: 85222     | TX bps: 84579576  | RX bps: 84539896  | TX error: 0         | RX error 0         | TX port error: 0         | RX queue error 0         |
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[root@a7s3 ~]#
[root@a7s3 ~]#
[root@a7s3 ~]# contrail-tools dpdkinfo -c
No. of forwarding lcores: 2
No. of interfaces: 5
Lcore 0:
        Interface: bond0.101           Queue ID: 0
        Interface: vhost0              Queue ID: 0
        Interface: tapd2d7bb67-c1      Queue ID: 0

Lcore 1:
        Interface: bond0.101           Queue ID: 1
        Interface: tap41a9ab05-64      Queue ID: 0  <---prox generator (single queue)
        Interface: tapd2d7bb67-c1      Queue ID: 1
kirankn80 commented 3 years ago

Core 2 is receiving the polling core. It receives all the traffic and load balances it to other cores. In this case, the only other core is core-1

pinggit commented 3 years ago

thanks, I thought of this. but I'm running vxlan traffic, shouldn't only MPLSoGRE traffic distributed between cores?

kirankn80 commented 3 years ago

This is the direction of VM - NIC, so encap does not apply, all traffic gets load balanced (unless the option --vr_no_load_balance) option is enabled..

pinggit commented 3 years ago

ok, let's use "lcore0" and "lcore1". instead of "lcore1" and "lcore2". so in this case lcore1 pulls the packet, distribute to lcore0. then I should see:

                                 lcore1                                                lcore0                                             lcore0
VM ----- RX port packets, TX queue packets  ----- RX queue packets, RX packet ---------- TX packets ----- bond

but I don't see RX port, TX queue packets on lcore1. instead, I only see RX queue packets on lcore1?

[root@a7s3 ~]# vif --get 3 --core 10
Vrouter Interface Table

Flags: P=Policy, X=Cross Connect, S=Service Chain, Mr=Receive Mirror
       Mt=Transmit Mirror, Tc=Transmit Checksum Offload, L3=Layer 3, L2=Layer 2
       D=DHCP, Vp=Vhost Physical, Pr=Promiscuous, Vnt=Native Vlan Tagged
       Mnp=No MAC Proxy, Dpdk=DPDK PMD Interface, Rfl=Receive Filtering Offload, Mon=Interface is Monitored
       Uuf=Unknown Unicast Flood, Vof=VLAN insert/strip offload, Df=Drop New Flows, L=MAC Learning Enabled
       Proxy=MAC Requests Proxied Always, Er=Etree Root, Mn=Mirror without Vlan Tag, HbsL=HBS Left Intf
       HbsR=HBS Right Intf, Ig=Igmp Trap Enabled

vif0/3      PMD: tap41a9ab05-64 NH: 38
            Type:Virtual HWaddr:00:00:5e:00:01:00 IPaddr:192.168.1.104
            Vrf:2 Mcast Vrf:2 Flags:L3L2DEr QOS:-1 Ref:12
            RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 0
            Core 10 RX packets:955345  bytes:57320682 errors:0
            Core 10 TX packets:1  bytes:42 errors:0
            Drops:18660668

[root@a7s3 ~]# vif --get 3 --core 11
Vrouter Interface Table

Flags: P=Policy, X=Cross Connect, S=Service Chain, Mr=Receive Mirror
       Mt=Transmit Mirror, Tc=Transmit Checksum Offload, L3=Layer 3, L2=Layer 2
       D=DHCP, Vp=Vhost Physical, Pr=Promiscuous, Vnt=Native Vlan Tagged
       Mnp=No MAC Proxy, Dpdk=DPDK PMD Interface, Rfl=Receive Filtering Offload, Mon=Interface is Monitored
       Uuf=Unknown Unicast Flood, Vof=VLAN insert/strip offload, Df=Drop New Flows, L=MAC Learning Enabled
       Proxy=MAC Requests Proxied Always, Er=Etree Root, Mn=Mirror without Vlan Tag, HbsL=HBS Left Intf
       HbsR=HBS Right Intf, Ig=Igmp Trap Enabled

vif0/3      PMD: tap41a9ab05-64 NH: 38
            Type:Virtual HWaddr:00:00:5e:00:01:00 IPaddr:192.168.1.104
            Vrf:2 Mcast Vrf:2 Flags:L3L2DEr QOS:-1 Ref:12
            Core 11 RX queue  packets:960705 errors:0
            RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 0
            Core 11 RX packets:710  bytes:29820 errors:0
            Core 11 TX packets:645  bytes:27090 errors:0
            Drops:18660668
ldurandadomia commented 3 years ago

RX port info is not provided in Contrail R2008. This is a regression. It will be restored in next release.

You have: OUTGOING PACKETS: VM (eth0 TX) ----> vif 0/3 RX (connected to VM) ---> polling thread on core 1 ---> processing thread on ??? ---> vif 0 TX --> physical ports If you want to know where are processed your packets, you have to look at vif0 TX counters.

INCOMING PACKETS: physical ports --> vif 0 RX ---> polling thread on ???? (probably 2 as you are not using GRE) ---> processing thread on core 2 ---> vif 0/3 TX (connected to VM) ---> VM (eth0 RX)

|Core 1 | TX pps: 0 | RX pps: 85221 ---> RX = polled on vif 0/3 |Core 2 | TX pps: 85262 | RX pps: 1 ---> TX = processed to be delivered on vif 0/3

Here core 1 and 2 are confusing (they are lcore 11 and 12 ...)

Laurent

pinggit commented 3 years ago

it does not look correct, here is another capture:

1 UDP flow, one direction (PROX), 64 Bytes

    [root@a7s3 ~]# vif --clear

    Vif stats cleared successfully on all cores for all interfaces

    [root@a7s3 ~]# contrail-tools dpdkvifstats.py -v 0 -c 2
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    |Core 1  | TX pps: 1512      | RX pps: 2         | TX bps: 166320    | RX bps: 132       | TX error: 0         | RX error 0         | TX port error: 0         | RX queue error 0         |
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    |Core 2  | TX pps: 1         | RX pps: 1         | TX bps: 112       | RX bps: 110       | TX error: 0         | RX error 0         | TX port error: 0         | RX queue error 0         |
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    |Total   | TX pps: 1513      | RX pps: 3         | TX bps: 1331456   | RX bps: 1936      | TX error: 0         | RX error 0         | TX port error: 0         | RX queue error 0         |
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    [root@a7s3 ~]#
    [root@a7s3 ~]#
    [root@a7s3 ~]# contrail-tools dpdkvifstats.py -v 3 -c 2
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    |Core 1  | TX pps: 0         | RX pps: 1504      | TX bps: 0         | RX bps: 90240     | TX error: 0         | RX error 0         | TX port error: 0         | RX queue error 0         |
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    |Core 2  | TX pps: 1         | RX pps: 1         | TX bps: 42        | RX bps: 56        | TX error: 0         | RX error 0         | TX port error: 0         | RX queue error 0         |
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    |Total   | TX pps: 1         | RX pps: 1505      | TX bps: 336       | RX bps: 722368    | TX error: 0         | RX error 0         | TX port error: 0         | RX queue error 0         |
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

    [root@a7s3 ~]# contrail-tools dpdkinfo -c
    No. of forwarding lcores: 2
    No. of interfaces: 5
    Lcore 0:
            Interface: bond0.101           Queue ID: 0
            Interface: vhost0              Queue ID: 0
            Interface: tapd2d7bb67-c1      Queue ID: 0

    Lcore 1:
            Interface: bond0.101           Queue ID: 1
            Interface: tap41a9ab05-64      Queue ID: 0
            Interface: tapd2d7bb67-c1      Queue ID: 1

    [root@a7s3 ~]# vif --get 3 --core 10
    Vrouter Interface Table

    vif0/3      PMD: tap41a9ab05-64 NH: 38
                Type:Virtual HWaddr:00:00:5e:00:01:00 IPaddr:192.168.1.104
                Vrf:2 Mcast Vrf:2 Flags:L3L2DEr QOS:-1 Ref:12
                RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 0
                Core 10 RX packets:31272  bytes:1876320 errors:0
                Core 10 TX packets:0  bytes:0 errors:0
                Drops:18660668

    [root@a7s3 ~]# vif --get 3 --core 11
    Vrouter Interface Table

    vif0/3      PMD: tap41a9ab05-64 NH: 38
                Type:Virtual HWaddr:00:00:5e:00:01:00 IPaddr:192.168.1.104
                Vrf:2 Mcast Vrf:2 Flags:L3L2DEr QOS:-1 Ref:12
                Core 11 RX queue  packets:35384 errors:0
                RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 0
                Core 11 RX packets:26  bytes:1092 errors:0
                Core 11 TX packets:24  bytes:1008 errors:0
                Drops:18660668

    [root@a7s3 ~]# vif --get 0 --core 10
    Vrouter Interface Table

    vif0/0      PCI: 0000:00:00.0 (Speed 20000, Duplex 1) NH: 4
                Type:Physical HWaddr:90:e2:ba:c3:af:20 IPaddr:0.0.0.0
                Vrf:0 Mcast Vrf:65535 Flags:TcL3L2VpVofEr QOS:-1 Ref:18
                Core 10 RX device packets:199  bytes:49057 errors:0
                RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 0
                Fabric Interface: eth_bond_bond0  Status: UP  Driver: net_bonding
                Slave Interface(0): 0000:02:00.0  Status: UP  Driver: net_ixgbe
                Slave Interface(1): 0000:02:00.1  Status: UP  Driver: net_ixgbe
                Vlan Id: 101  VLAN fwd Interface: vfw
                Core 10 RX packets:131  bytes:37595 errors:0
                Core 10 TX packets:48756  bytes:5362888 errors:0
                Drops:0
                Core 10 TX device packets:49024  bytes:5730372 errors:0

    [root@a7s3 ~]# vif --get 0 --core 11
    Vrouter Interface Table

    vif0/0      PCI: 0000:00:00.0 (Speed 20000, Duplex 1) NH: 4
                Type:Physical HWaddr:90:e2:ba:c3:af:20 IPaddr:0.0.0.0
                Vrf:0 Mcast Vrf:65535 Flags:TcL3L2VpVofEr QOS:-1 Ref:18
                RX queue errors to lcore 0 0 0 0 0 0 0 0 0 0 0 0
                Fabric Interface: eth_bond_bond0  Status: UP  Driver: net_bonding
                Slave Interface(0): 0000:02:00.0  Status: UP  Driver: net_ixgbe
                Slave Interface(1): 0000:02:00.1  Status: UP  Driver: net_ixgbe
                Vlan Id: 101  VLAN fwd Interface: vfw
                Core 11 RX packets:67  bytes:9860 errors:0
                Core 11 TX packets:181  bytes:162062 errors:0
                Drops:0
pinggit commented 3 years ago

this is from VM to bond, so LB will always happen. there is also any issue on LB from bond to VM though. https://contrail-jws.atlassian.net/browse/CEM-19837