search

github / glb-director

GitHub Load Balancer Director and supporting tooling.
Other
2.37k stars 227 forks source link

glb-director-xdp on bonded nic #106

Closed jstangroome closed 4 years ago

jstangroome commented 4 years ago

I have successfully deployed glb-director-xdp v1.0.6 on a Debian 10 server (a t1.small/x86 instance from Packet).

Packet servers have bonded nics:

$ ip -o link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000\    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: enp0s20f0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 xdpgeneric qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000\    link/ether 0c:c4:7a:81:09:9c brd ff:ff:ff:ff:ff:ff\    prog/xdp id 368 tag 631855c97cb7abd1
3: enp0s20f1: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc mq master bond0 state DOWN mode DEFAULT group default qlen 1000\    link/ether 0c:c4:7a:81:09:9d brd ff:ff:ff:ff:ff:ff
4: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 xdpgeneric qdisc noqueue state UP mode DEFAULT group default qlen 1000\    link/ether 0c:c4:7a:81:09:9c brd ff:ff:ff:ff:ff:ff\    prog/xdp id 351 tag 631855c97cb7abd1

I initially tried using the xdp-root-shim@bond0 service with /etc/default/glb-director-xdp containing:

GLB_DIRECTOR_XDP_ROOT_PATHS="--xdp-root-path=/sys/fs/bpf/xdp_root_array@bond0"

with no success.

Then I tried configuring xdp on the underlying nic instead of bond0 via:

GLB_DIRECTOR_XDP_ROOT_PATHS="--xdp-root-path=/sys/fs/bpf/xdp_root_array@enp0s20f0"

And this works.

However, I'd like to have glb-director-xdp handling traffic from both underlying nics. Is there a way to configure this?

theojulienne commented 4 years ago

However, I'd like to have glb-director-xdp handling traffic from both underlying nics. Is there a way to configure this?

Yes! We are doing exactly this, you can bring up the xdp-root-shim once for each NIC and then add the --xdp-root-path command line option twice to bind to both interfaces at the same time:

GLB_DIRECTOR_XDP_ROOT_PATHS="--xdp-root-path=/sys/fs/bpf/xdp_root_array@enp0s20f0 --xdp-root-path=/sys/fs/bpf/xdp_root_array@enp0sblah"

Apologies that the documentation is pretty lacking for the XDP version - it's still under development and I'll be making a pass over it soon to make it more complete in terms of monitoring & documentation.

jstangroome commented 4 years ago

This works exactly as described. Closing.