Question: Network Adaptor Problem in VirtualBox

[JackKuo-tw]

I use Vagrant with VirtualBox and encounter some problems while executing the installation.

If I choose virtio as the network adaptor, the error seems to be the adaptor not support RSS: (But it finished the DPDK helloworld)

Thus, I change the adaptor to Intel 82545EM, which is supported by DPDK, but stuck at DPDK helloworld test

That's weird, since Intel 82545EM can pass DPDK compilation and show that DPDK supports it:

Here is my Vagrantfile and bootstrap.sh

# -*- mode: ruby -*-
# vi: set ft=ruby :
[
  { :name => "vagrant-reload", :version => ">= 0.0.1" },
  { :name => "vagrant-cachier", :version => ">= 1.2.1"}
].each do |plugin|

  if not Vagrant.has_plugin?(plugin[:name], plugin[:version])
    raise "#{plugin[:name]} #{plugin[:version]} is required. " \
          "Please run `vagrant plugin install #{plugin[:name]}`"
  end
end

Vagrant.configure("2") do |config|
  config.vm.box = "bento/ubuntu-20.04"
  config.vm.hostname = "opennetvm"

  if Vagrant.has_plugin?("vagrant-cachier")
    config.cache.scope = "machine"
  end

  config.vm.provision :shell, path: "bootstrap.sh", privileged: false
  config.vm.provision :reload

  config.vm.provider "virtualbox" do |vb|
    # Uncomment below if necessary.
    # vb.gui = true
    vb.name = "OpenNetVM Server"

    # Customize below. At least two or more vCPUs are recommended.
    vb.cpus = 6
    vb.memory = 4096

    vb.customize ["modifyvm", :id, "--nic1", "nat"]
    vb.customize ["modifyvm", :id, "--nic2", "natnetwork"]
    vb.customize ["modifyvm", :id, "--nic3", "natnetwork"]
    vb.customize ["modifyvm", :id, "--nictype1", "virtio"]
#    vb.customize ["modifyvm", :id, "--nictype2", "82545EM"]
#    vb.customize ["modifyvm", :id, "--nictype3", "82545EM"]
    vb.customize ["modifyvm", :id, "--nictype2", "virtio"]
    vb.customize ["modifyvm", :id, "--nictype3", "virtio"]
    vb.customize ["modifyvm", :id, "--nicpromisc2", "allow-all"]
    vb.customize ["modifyvm", :id, "--nat-network2", "onv"]
    vb.customize ["modifyvm", :id, "--nicpromisc3", "allow-all"]
    vb.customize ["modifyvm", :id, "--nat-network3", "onv"]

    # without this, spinlock and rdtsc can be significantly slower...
    vb.customize ["modifyvm", :id, "--paravirtprovider", "kvm"]

    # Uncomment this if you want AVX2...
    #vb.customize ["setextradata", :id, "VBoxInternal/CPUM/IsaExts/AVX2", "1"]
  end
end

sudo apt-get update
sudo apt-get install -y build-essential linux-headers-$(uname -r) git bc
sudo apt-get install -y python3 python
sudo apt-get install -y libnuma-dev
sudo apt-get install -y gcc make pkg-config

# Setup repo
git clone https://github.com/sdnfv/openNetVM
cd openNetVM
git checkout master

git submodule sync
git submodule update --init

echo export ONVM_HOME=$(pwd) >> ~/.bashrc
echo export ONVM_HOME=$(pwd) >> ~vagrant/.bashrc

export ONVM_HOME=$(pwd) >> ~/.bashrc
export ONVM_HOME=$(pwd) >> ~vagrant/.bashrc

cd dpdk
echo export RTE_SDK=$(pwd) >> ~/.bashrc
echo export RTE_TARGET=x86_64-native-linuxapp-gcc  >> ~/.bashrc
echo export ONVM_NUM_HUGEPAGES=1024 >> ~/.bashrc
echo export ONVM_NIC_PCI=\" 00:08.0 00:09.0 \" >> ~/.bashrc

export RTE_SDK=$(pwd) >> ~/.bashrc
export RTE_TARGET=x86_64-native-linuxapp-gcc  >> ~/.bashrc
export ONVM_NUM_HUGEPAGES=1024 >> ~/.bashrc
export ONVM_NIC_PCI=" 00:08.0 00:09.0 " >> ~/.bashrc

sudo sh -c "echo 0 > /proc/sys/kernel/randomize_va_space"

# Configure & compile DPDK
cd $ONVM_HOME/scripts
./install.sh

# Run DPDK HelloWorld Application
cd $ONVM_HOME/dpdk/examples/helloworld
make
sudo ./build/helloworld -l 0,1 -n 1

# Make and test openNetVM
cd $ONVM_HOME/onvm
make
cd ..

[twood02]

A possible explanation is that your script above doesn't show you actually binding the NIC to the DPDK userspace driver. You can do that with our setup_environment.sh script (or with the DPDK scripts it calls).

If that doesn't resolve the issue, then it sounds like these are DPDK issues, not OpenNetVM specific. I suggest you contact the DPDK Users mailing list to see if they have suggestions. We have not attempted to run DPDK/ONVM inside virtual box, so I suspect the underlying issue is problems with the NIC support.

[JackKuo-tw]

@twood02 Thanks, I'll try to contact them. setup_environment.sh shows that my NIC has already bound to igb_uio, and that should be the DPDK driver.

[JackKuo-tw]

Actually, Intel 82545EM can pass helloworld, just take a very very long time. In my test, use time command:

• real: 4m46s
• user: 0m2s
• sys: 0m0.1s

also, it can run /onvm/go.sh after a very long time wait.

---

But if I use QEMU + KVM, everything is fine and can pass all the tests in installation and so fast. Here is my command: sudo qemu-system-x86_64 -smp 6 -hda ubuntu20.04.qcow2 -m 4g -vga virtio -accel kvm -nic tap,model=e1000 -nic tap,model=e1000 -cpu Skylake-Server

The virtio doesn't support RSS, so it fails /onvm/go.sh on QEMU, too.

[twood02]

It would not be too difficult to disable RSS in ONVM. We can add this as a feature if it would be helpful. ONVM uses the RSS value to support load balancing among NFs with the same service ID.

[JackKuo-tw]

That would be great for those who want to try it on common VM simulators. 👍 Do you mean modifying ETH_MQ_RX_RSS in onvm_init.c to ETH_MQ_RX_NONE? I did it and still failed...