search

cnplab / xennet

Xen optimizations for fast packet I/O
Other
11 stars 7 forks source link

Xen/ClickOS optimizations on igb/e1000 ? #4

Open ghost opened 9 years ago

ghost commented 9 years ago

Is there any way to install the optimizations (netmap+xen-netback) on machines with igb or e1000 drivers? I have built clickOS --with-netmap I have built and insmodded netmap_lin.ko, e1000.ko,xen-netback.ko clickOS seems to start, but no packets can be retrieved... This is because I am not using ixgbe drivers or something else is missing?

after insmodding the modules and after starting clickOS with cosmos create example.cfg, there is this: [ 123.350660] 151.759186 [1456] netmap_set_ringid deprecated API, old ringid 0x0 -> ringid 0 reg 1 [ 123.351288] 151.759816 [1049] netmap_mem_global_config reconfiguring [ 123.466124] 151.874633 [ 319] e1000_netmap_init_buffers i now is 255

[ 1963.443199] 525.925587 [2259] netmap_attach success for vif2.0 [ 1963.619724] 526.102112 [1456] netmap_set_ringid deprecated API, old ringid 0x0 -> ringid 0 reg 1

this is the console output after starting clickOS: (cosmos create example.cfg)

Thread "shutdown": pointer: 0x2002002fb0, stack: 0x210000 Dummy main: start_info=0x198040 Thread "main": pointer: 0x2002003760, stack: 0x220000 sparsing 0MB at 177000 "main" [on_status:205] router id 0 [on_status:206] status change to Running Thread "click": pointer: 0x2002010a00, stack: 0x240000 backend dom 0 Mapping TX rings 0 map errors Mapping RX rings 0 map errors Mapping TX buffers 0 map errors Mapping RX buffers 0 map errors init_netfront_netmap device/vif/0 Waiting for /local/domain/0/backend/vif/1/0/state change to connected unmasking event-channel-tx 4 unmasking event-channel-rx 5 [router_thread:157] Starting driver...

jpemartins commented 9 years ago

Hum, in principle it should work since netmap already has support for running under Xen. Did you attach the NIC to the vale bridge? It your vif attached to the same vale bridge?

Could you paste example.cfg?

Thanks!

ghost commented 9 years ago

Thx for the feedback, I do like this: vale-ctl -a vale0:eth0 I assume this attaches the NIC (eth0) to vale.

after starting clickos, checking with vale-ctl gives: bdg_ctl [98] bridge:0 port:0 vale0:eth0 bdg_ctl [98] bridge:0 port:1 vale0:vif4.0

this is example.cfg name = 'clickos' kernel = '/home/steven/clickos/minios/build/clickos_x86_64' vcpus = '1' # pinning your VCPU helps performance #cpus = '3' memory = '32' # uncoment this line if you wish to use our backend vif =['bridge=vale0,script=/etc/xen/scripts/vif-vale'] on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'preserve' click ='test-device.click'

this is the click script test-device.click: 'FromDevice -> Print(ok) -> Discard;'

When I use pkt-gen to generate some traffic on the same vale0 switch, (on the same machine where clickOS is running, in another terminal) it works and I can see the packets being printed in the clickOS console. pkt-gen -i vale0:10 -f tx

But when I try to send something from another machine to eth0, no packets are printed... so I assume I am not receiving anything on the NIC... This setup is done with several UbuntuServers as VM in Virtual Box. When using the xenbr0 switch (bridge-utils) instead of VALE, it also works.

Is there anything I'm missing to correctly attach the NIC (eth0) to VALE and clickOS?

jpemartins commented 9 years ago

Found it! So it looks like netmap submodule is pointing to the master branch which still doesn't contain my IOMMU/Xen patches. netmap branch next already has that. Thing is there are usually netmap API modifications which might breaking the frontends. Let me double check this and I get back to you.

Thanks a lot!

ghost commented 9 years ago

I tried some other things with the netmap-next branch in Xen Dom0 (UbuntuServer, not yet using any clickos VM): This can receive from another machine pkt-gen -i eth0 -f rx But if I attach eth0 to a VALE switch like this, it cannot receive anymore from another machine pkt-gen -i vale0:eth0 -f rx I suppose I do not need to explicitly connect the NIC (eth0) to the VALE switch using the netmap bridge command?

jpemartins commented 9 years ago

The switch port associated with the NIC (pkt-gen -i vale:eth0 -f rx) can't be directly used. You should get an error on pkt-gen actually.

You need to attach it first to the switch with vale-ctl -a vale:eth0 After that you just using another port name in the receiver/generator such as pkt-gen -i vale:0 -f rx. You might also need to enable promiscuous mode ifconfig eth0 promisc since pkt-gen sends 00:00: on source mac address.

Let me know if it still doesn't work.