Sync OpenStack build with VNF test configurations

[robertstarmer]

• [x] Update Grub config
• [x] Updated VPP startup config
• [x] Update default MTU to 9200
• [x] Add 3 core flavor to defaults 'vnf.3c'
• [x] Add to the default flavor policy to try to replace the virsh cpu pinning commands
• [ ] Possibly set workers statically in VPP startup config

Related to Issue #199

Grub delta:

isolcpus=2,4,30,32 nohz_full=2-27,30-55 rcu_nocbs=2-27,30-55

Vpp startup.conf

unix {
  nodaemon
  log /var/log/vpp/vpp.log
  full-coredump
  cli-listen /run/vpp/cli.sock
  gid vpp
}

api-trace {
  on
}

api-segment {
  gid vpp
}

cpu {
  main-core 0
  corelist-workers 2,4,30,32
  heapsize {
   2G
  }
}

dpdk {
  dev default {
    num-rx-queues 4
  }
  no-multi-seg
  num-mbufs 65536
  no-tx-checksum-offload
  huge-dir /dev/hugepages
  socket-mem 2048,2048
}

Add to VPP startup config:

set int mtu 9200 TenGigabitEthernet5e/0/1

Add flavor model for cpu pinning and verify equivalent configuration:

virsh vcpupin <ID> 0 <Main Core>
virsh vcpupin <ID> 1 <Worker Core 1>
virsh vcpupin <ID> 2 <Worker Core 2>
virsh emulatorpin <ID> <Main Core>,<Worker Core 1>,<Worker Core 2>
# numatune not needed (already strict numa 0)

Add to the vnf.3c flavor:

--property hw:cpu_policy=dedicated --property hw:cpu_thread_policy=isolate

Work setup, configure on startup rather than via script if possible

#! /bin/bash

in_use=( 0 1 2 3 7 9 11 12 13 14 15 16 )
not_in_use=( 4 5 6 8 10 17 )
worker=0
for i in ${in_use[@]}; do
  vppctl set interface rx-placement VirtualEthernet0/0/${i} queue 0 worker ${worker}
  worker=$((($worker + 1) % 4))
done

worker=0
for i in ${not_in_use[@]}; do
  vppctl set interface rx-placement VirtualEthernet0/0/${i} queue 0 worker ${worker}
  worker=$((($worker + 1) % 4))
done

[taylor]

@robertstarmer @bvera please document the current state for this item:

Possibly set workers statically in VPP startup config

[wverac]

@robertstarmer @bvera please document the current state for this item:

Possibly set workers statically in VPP startup config

Automating it will require checking what interfaces are used for external traffic, inspecting every VM through virsh and create the arrays (in_use, not_in_use) with the correct information.

[michaelspedersen]

A few weeks ago I updated the script I have been using to do rx-queue pinning (didn't realize there was this issue that I could have referenced). It works similar to what @bvera mentions in his comment: repin_rxqs.sh

One thing that makes it a little more difficult is that the static MACs given when creating the ports in Openstack do not seem to propagate correctly to VPP. As a result I am using UID values for finding the "correct" interfaces in each case.