search

pinggit / dpdk-contrail-book

contrail dpdk day one book
9 stars 3 forks source link

fwding core default cpu pinning #20

Closed pinggit closed 3 years ago

pinggit commented 3 years ago

is there a default cpu pinning? in this setup right now there is no pinning.

[root@a7s3 ~]# cat /etc/sysconfig/network-scripts/ifcfg-vhost0
NETMASK=255.255.255.0
BOOTPROTO=none
DEVICE=vhost0
IPADDR=8.0.0.4
ONBOOT=yes
TYPE=dpdk
NM_CONTROLLED=no
BIND_INT=0000:00:00.0

but why we have 8 fwding lcores running and each got pinned to a dedicated CPU?

[root@a7s3 ~]# ps -Tp `pidof contrail-vrouter-dpdk`
  PID  SPID TTY          TIME CMD
 3706  3706 ?        04:16:19 contrail-vroute
 3706  3761 ?        00:05:02 eal-intr-thread
 3706  3762 ?        00:00:00 rte_mp_handle
 3706  3785 ?        05:30:00 lcore-slave-1
 3706  3786 ?        00:00:00 lcore-slave-2
 3706  3787 ?        00:00:13 lcore-slave-8
 3706  3788 ?        00:04:43 lcore-slave-9
 3706  3789 ?        7-10:47:51 lcore-slave-10
 3706  3790 ?        7-10:47:43 lcore-slave-11
 3706  3791 ?        7-10:46:51 lcore-slave-12
 3706  3792 ?        7-05:18:31 lcore-slave-13
 3706  3793 ?        7-10:48:01 lcore-slave-14
 3706  3794 ?        7-10:46:54 lcore-slave-15
 3706  3795 ?        7-10:50:55 lcore-slave-16
 3706  3796 ?        7-10:50:08 lcore-slave-17
 3706  4873 ?        00:00:00 lcore-slave-9

[root@a7s3 ~]# taskset -cpa `pidof contrail-vrouter-dpdk`
pid 3706's current affinity list: 0-11
pid 3761's current affinity list: 0-11
pid 3762's current affinity list: 0-11
pid 3785's current affinity list: 0-11
pid 3786's current affinity list: 0-11
pid 3787's current affinity list: 0-11
pid 3788's current affinity list: 0-11
pid 3789's current affinity list: 1
pid 3790's current affinity list: 2
pid 3791's current affinity list: 3
pid 3792's current affinity list: 4
pid 3793's current affinity list: 5
pid 3794's current affinity list: 6
pid 3795's current affinity list: 7
pid 3796's current affinity list: 8
pid 4873's current affinity list: 0-11
kirankn80 commented 3 years ago

In centOS kolla, the pinning is present here -

[root@a7s4-kiran ~]# cat /etc/contrail/common_vrouter.env |grep -i cpu CPU_CORE_MASK=0x1fe

ldurandadomia commented 3 years ago

Pinning is specified in installation tool. There is two place to specify vrouter CPU pinning (CPU_LIST, and other variables). They are: /etc/sysconfig/network-scripts/ifcfg-vhost0 /etc/contrail/common_vrouter.env

In general, the used Contrail deployer (TripleO or Ansible) is putting these variables into common_vrouter.env. ifcfg-vhost0, is used by end-user to perform changes after vrouter deployment. But there are some exceptions ....

This is why, the best is to check both files.

If CPU_LIST is not specified, there is probably a default value (to be checked - Could be 0x1 --> it means CPU1 - CPU0 Kept for the OS - it's also possible there is a default value in each installer). But this is not good to keep DPDK default setup without specifying:

pinggit commented 3 years ago

this is tested and works fine in kolla/centos:

    cat /etc/contrail/common_vrouter.env |grep -i cpu 
        CPU_CORE_MASK=0x1fe
    cd /etc/contrail/vrouter/
    docker stop vrouter_vrouter-agent-dpdk_1 && docker rm vrouter_vrouter-agent-dpdk_1 && docker-compose up -d