Open G-Venkata-Surya-Sai opened 1 year ago
AIO VM playing the role of the hypervisor.
Network settings in the VM:
root@faucet-chewie-aio:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:20:aa:0d brd ff:ff:ff:ff:ff:ff
altname enp0s3
inet 192.168.51.79/23 brd 192.168.51.255 scope global ens3
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fe20:aa0d/64 scope link
valid_lft forever preferred_lft forever
3: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:dc:b8:79 brd ff:ff:ff:ff:ff:ff
altname enp0s4
inet 192.168.47.79/23 brd 192.168.47.255 scope global ens4
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fedc:b879/64 scope link
valid_lft forever preferred_lft forever
4: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovs-system state UP group default qlen 1000
link/ether 52:54:00:c8:20:59 brd ff:ff:ff:ff:ff:ff
altname enp0s5
inet 10.200.0.79/22 brd 10.200.3.255 scope global ens5
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fec8:2059/64 scope link
valid_lft forever preferred_lft forever
5: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 76:39:8b:6d:e0:cd brd ff:ff:ff:ff:ff:ff
6: br0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 72:93:ba:45:a2:42 brd ff:ff:ff:ff:ff:ff
7: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:5d:30:ee:76 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
valid_lft forever preferred_lft forever
8: br-2d6c67783f7d: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:b7:fa:d9:7f brd ff:ff:ff:ff:ff:ff
inet 172.24.0.1/16 brd 172.24.255.255 scope global br-2d6c67783f7d
valid_lft forever preferred_lft forever
9: dm-857f893be141: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default
link/ether 0a:1b:45:a9:90:82 brd ff:ff:ff:ff:ff:ff
inet6 fe80::b88a:faff:fe64:ec6b/64 scope link
valid_lft forever preferred_lft forever
43: vethd335b8dc@if42: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
link/ether e2:dc:88:d7:05:f3 brd ff:ff:ff:ff:ff:ff link-netnsid 0
45: vethac0ae1e2@if44: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
link/ether 86:48:33:0e:a6:8d brd ff:ff:ff:ff:ff:ff link-netnsid 1
47: veth551adc11@if46: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
link/ether 12:07:d1:7c:20:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 2
root@faucet-chewie-aio:~# ip r
default via 192.168.50.1 dev ens3 proto static
10.200.0.0/22 dev ens5 proto kernel scope link src 10.200.0.79
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
172.24.0.0/16 dev br-2d6c67783f7d proto kernel scope link src 172.24.0.1 linkdown
192.168.46.0/23 dev ens4 proto kernel scope link src 192.168.47.79
192.168.50.0/23 dev ens3 proto kernel scope link src 192.168.51.79
The three containers are
root@faucet-chewie-aio:~# lxc ls
+-------+---------+--------------------+------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+-------+---------+--------------------+------+-----------+-----------+
| cont0 | RUNNING | | | CONTAINER | 0 |
+-------+---------+--------------------+------+-----------+-----------+
| cont1 | RUNNING | 10.200.3.226 (cn0) | | CONTAINER | 0 |
+-------+---------+--------------------+------+-----------+-----------+
| cont2 | RUNNING | 10.200.3.227 (cn0) | | CONTAINER | 0 |
+-------+---------+--------------------+------+-----------+-----------+
These containers are connected to ovs-bridge named br0.
root@faucet-chewie-aio:~# ovs-vsctl show
be4fcfbf-7f49-477f-9159-1f2517ad78cb
Bridge br0
Controller "tcp:127.0.0.1:6654"
is_connected: true
Controller "tcp:127.0.0.1:6653"
is_connected: true
fail_mode: secure
Port ens5
Interface ens5
Port vethd335b8dc
Interface vethd335b8dc
Port br0
Interface br0
type: internal
Port vethac0ae1e2
Interface vethac0ae1e2
Port veth551adc11
Interface veth551adc11
ovs_version: "2.15.0"
root@faucet-chewie-aio:~# ovs-dpctl show
system@ovs-system:
lookups: hit:1851490 missed:10026 lost:0
flows: 3
masks: hit:1869147 total:2 hit/pkt:1.00
port 0: ovs-system (internal)
port 1: br0 (internal)
port 2: vethd335b8dc
port 3: vethac0ae1e2
port 4: veth551adc11
port 5: ens5
Faucet is running locally in the AIO VM.
root@faucet-chewie-aio:~# systemctl status faucet
● faucet.service - "Faucet OpenFlow switch controller"
Loaded: loaded (/lib/systemd/system/faucet.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2022-12-07 08:25:39 IST; 4h 18min ago
Docs: https://docs.faucet.nz
Main PID: 63796 (osken-manager)
Tasks: 1 (limit: 9508)
Memory: 68.0M
CPU: 25.751s
CGroup: /system.slice/faucet.service
└─63796 /usr/bin/python3 /usr/bin/osken-manager --config-file=/etc/faucet/ryu.conf --ofp-tcp-listen-port=6653 faucet.faucet
root@faucet-chewie-aio:~# netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:6653 0.0.0.0:* LISTEN 63796/python3
tcp 0 0 0.0.0.0:6654 0.0.0.0:* LISTEN 1161/python3
tcp 0 0 127.0.0.1:41903 0.0.0.0:* LISTEN 685/containerd
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/init
tcp 0 0 0.0.0.0:9302 0.0.0.0:* LISTEN 63796/python3
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 686/sshd: /usr/sbin
tcp 0 0 0.0.0.0:9303 0.0.0.0:* LISTEN 1161/python3
tcp6 0 0 :::9090 :::* LISTEN 501/prometheus
tcp6 0 0 :::9100 :::* LISTEN 498/prometheus-node
tcp6 0 0 :::111 :::* LISTEN 1/init
tcp6 0 0 :::22 :::* LISTEN 686/sshd: /usr/sbin
tcp6 0 0 :::3000 :::* LISTEN 1162/grafana-server
udp 0 0 0.0.0.0:51948 0.0.0.0:* 63796/python3
udp 0 0 0.0.0.0:111 0.0.0.0:* 1/init
udp6 0 0 :::111 :::* 1/init
Authentication server:
Radius server is present outside the VM. with IP 192.168.50.7. Radius server is working fine. I tested this using radtest with the above user id(user1) and password(topsecret).
root@faucet-chewie-aio:~# radtest -x user1 topsecret 192.168.50.7 -0 testing123 radiussrv
Sent Access-Request Id 142 from 0.0.0.0:55817 to 192.168.50.7:1812 length 84
User-Name = "user1"
User-Password = "topsecret"
NAS-IP-Address = 127.0.1.1
NAS-Port = 0
Message-Authenticator = 0x00
Framed-Protocol = PPP
Cleartext-Password = "topsecret"
Received Access-Accept Id 142 from 192.168.50.7:1812 to 192.168.51.79:55817 length 32
Framed-Protocol = PPP
Framed-Compression = Van-Jacobson-TCP-IP
Supplicant:
Cont0 is the EAPOL supplicant, generating EAPOL packets from the netplan file.
root@cont0:~# vi /etc/netplan/10-lxc.yaml
network:
version: 2
ethernets:
cn0:
auth:
key-management: 802.1x
method: ttls
identity: "user1"
password: "topsecret”
dhcp4: true
Authenticator:
Faucet/chewie act as authenticator. Faucet is running locally inside the VM at 127.0.0.1.
root@faucet-chewie-aio:~# vi /etc/faucet/faucet.yaml
---
vlans:
office:
vid: 560
description: "overlay host network"
dps:
sw1:
dot1x:
radius_ip: "192.168.50.7"
radius_port: 1812
radius_secret: "testing123"
nfv_intf: "ens3"
nfv_sw_port: 1
dp_id: 0x1
hardware: "Open vSwitch"
interfaces:
1:
name: "DMZ"
description: "DMZ"
output_only: true
4:
name: "ens5"
description: "trunk"
native_vlan: office
18:
name: "cont0"
description: "cont0"
native_vlan: office
dot1x: true
19:
name: "cont1"
description: "cont1"
native_vlan: office
20:
name: "cont2"
description: "cont2"
native_vlan: office
I got these port numbers 4,18,19,20 from the “/var/log/faucet/faucet.log” file. Before adding dot1x, all these containers are getting DHCP IP. So we can confirm that these ports are correct. I arbitrarily assigned nfv_sw_port to 1.
Results:
Collected packet traces in port 18, ens3, ens5, lo
On Port 18(Port connected to cont0): I could notice EAPOL-start, but there is no further handshake.
On ens3 Port: We could see RADIUS server fails with Access-Reject. It failed because the username value is not the same as what we are sending. Username inside RADIUS packet is some arbitrary mac address “008e732690c7” instead of “user1”.
On port ens5, there are no EAPOL and radius packets. On port lo, there are openflow messages. But there are no EAPOL, RADIUS messages.
Issues:
I found a similar issue to yours as well but in my case Faucet also terminates. Below is the error message as well as faucet terminating due to the failed authentication between radius and chewie
Motive:
At our campus data-center we are implementing zero-trust network architecture, where each of the instances, like bare-metal servers, VMs/containers running on hypervisors (QEMU-KVM/LXD), have to go through network access control (802.1x) to get Internet connectivity. For bare-metal servers, we have Altoline Whitebox switches that can run Open Virtual Switch (OVS). For VMs and Containers, OVS will be running inside hypervisors. We are experimenting with Faucet SDN, which already has the experimental 802.1x feature (Chewie).
Description: while verifying 802.1x port-based authentication using faucet/chewie, we notice arbitrary mac address in the username field of RADIUS messages to the radius server.
Prototype setup: This prototype setup is inside an All-In-One VM named “faucet-chewie-aio” with Debian 11 operating system. Inside this VM, we are running 3 LXD containers named cont0, cont1, and cont2, all of them running ubuntu 22.04 LTS server (headless systems).