Forwarding traffic with clickos

[lmarcuzzo]

Hi, im trying to use clickos to forward traffic betwenn 2 VM, but am having issues.

Each vm is connected with one bridge(br0 and br1) and click interconnects them(clickos is connected both with br0 and br1), like this:

VM1 <-br0-> clickos <-br1-> VM2

My clickos config is:

kernel = './clickos_x86_64' vcpus = '1'

cpus = '2'

memory = '256' vif = ['bridge=br0', 'bridge=br1'] name = 'click0' on_crash = 'preserve'

And the click config running is:

FromDevice(0) -> Print("De 0 para 1") -> ToDevice(1); FromDevice(1) -> Print("De 1 para 0") -> ToDevice(0);

I can ping between the vms, but when i try using iperf, or telnet, i cant get a connection between then. When i ping between then, this is shown on console:

From port to port De 1 para 0: 98 | 00163e64 e4ff0016 3eea4c24 08004500 0054b221 40004001 De 0 para 1: 98 | 00163eea 4c240016 3e64e4ff 08004500 00540efb 00004001 De 1 para 0: 98 | 00163e64 e4ff0016 3eea4c24 08004500 0054b29b 40004001 De 0 para 1: 98 | 00163eea 4c240016 3e64e4ff 08004500 00540f6c 00004001

And when i try other program(using both TCP and UDP), it shows:

De 1 para 0: 66 | 00163e64 e4ff0016 3eea4c24 08004500 0034e9fe 40004006 De 1 para 0: 66 | 00163e64 e4ff0016 3eea4c24 08004500 0034e9ff 40004006 De 1 para 0: 66 | 00163e64 e4ff0016 3eea4c24 08004500 0034ea00 40004006 De 1 para 0: 42 | 00163e64 e4ff0016 3eea4c24 08060001 08000604 00010016 De 0 para 1: 42 | 00163eea 4c240016 3e64e4ff 08060001 08000604 00020016

Running the same click config on a linux VM using click linux module instead of clickos i have no issues.

[fmanco]

Hi

Can you provide pcap traces on VM1 and VM2 interfaces?

Thank you.

[lmarcuzzo]

Hi, i ran a few more tests and put the pcaps on a zip.

I tested a ping, telnet connection and nmap (using TCP SYN scan, TCP connect scan and udp)on clickOS and linux running click as kernel module(TCP connect and udp on linux is not in the pcap, but it works).

It seems doing with telnet and TCP connect scan with clickOS, a TCP packet arrives at VM2, but no response is sent back. On click running on linux the TCP arrives and the response is sent.

And for some reason using TCP SYN scan through clickOS works, but telnet, TCP connect or UDP still doens't.

Some more info that may be useful: Im running xen 4.4.3 on Fedora 21 and using linux bridges. Both VMs are running paravirtualized Debian 8 with firewalls disabled. IPs on VMs are static.

Thank you for your help. tests.zip

[jpemartins]

Chances are that it is because of checksum offloading which IIRC wasnt correctly handled well between end boxes. could you try with all offloadings disabled on your vms? gso/gro isnt implemented too so packets fragmented across slots wont work (that is after 3whs is working a paged buffer wouldnt work)

[lmarcuzzo]

Hi, seems that checksum offloading was the issue. I disabled checksum and gso/gro from inside the vms(ethX) and from the hypervisor(disabling on vifX.X).

I used this command:

ethtool -K eth0 rx off tx off sg off tso off ufo off gso off gro off lro off

Thanks for your help.