Traffic-shaping questions

[borisoks]

Hi Patrick,

We are setting up a mobile lab with iOS and Android 10-20 devices. Raspberry Pi's running demonized python wptagents have been super stable! For traffic shaping we created wifi access point (AP) on each Raspberry Pi, which is used by its corresponding device. We are using wondershaper (tc) for on demand throttling of Raspberry Pi's AP. We have run into two issues with this setup: 1. Raspberry Pi wireless AP hit a ceiling of 5-10 Mbps. 2. Devices loose wifi connection to the AP every 12 hours or so and do not automatically reconnect.

We have come to conclusion that our best bet is to go with setup similar to what you documented - FreeBSD bridge + Airport, etc. We do not have expertise building FreeBSD servers or a rack space for the blade. Do you know where we can purchase an out of the box desktop/mini machine with with 2+ network interfaces and pre-installed FreeBSD OS?

Thank you, Boris

[pmeenan]

Not sure if it helps but I'm hoping to implement reverse tethering for iOS in the next week or two which will eliminate the WiFi need. Android should work with Android <= 6 and I'm also hoping to implement a reverse-tether solution that will work 5-9+ but that will be a little longer.

I'm not aware of any pre-built systems. As far as hardware goes, I am using an older version of one of these. The Atom CPU's are more than enough for traffic-shaping and the built-in BMC makes remote management/OS install much easier.

As far as FreeBSD install goes, it is a super-basic install with just the kernel and sshd. I am probably several kernels back so you just need to make sure that ipfw ins included in case they moved to a different firewall stack. There is some notes on the config here but I can pull the exact init script from the server which has all of the config for bridging and the pipes if that would help.

[bbraz]

For the wifi connection issue on android, this might help:

Disabling wifi auto disconnect on Android device
 If the wifi access point you are using does not have an internet connection, the Android devices will not reconnect to the wifi access point upon subsequent reboots. To resolve this, you need to run the following command to disable this feature on the device:

adb shell su -c "settings put global captive_portal_detection_enabled 0"

More details at: https://github.com/WPO-Foundation/wptagent/issues/135

[borisoks]

Thank you @pmeenan for the info, we are likely going to go with the hardware you recommended. Can you please send me the init script from the server.

[borisoks]

Thank you @bbraz I will try your suggestion.

[pmeenan]

/etc/rc.conf:

hostname="bridge"
sshd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="NO"

Basically the only thing of importance there is the sshd_enable to enable ssh access on the system.

/etc/rc.conf.local:

#BRIDGE CONFIG
cloned_interfaces="bridge0"
ifconfig_bridge0="addm em0 addm em1"
ifconfig_em1="up"
ifconfig_em0="inet 192.168.0.5/16"
defaultrouter="192.168.0.1"

#IPFW Config
firewall_enable="YES"
firewall_type="open"
firewall_script="/etc/ipfw.rules"

That creates a bridge across the 2 ethernet interfaces (em0 and em1), assigns a management address to the em0 interface for ssh access and rins the firewall script.

/etc/ipfw.rules:

ipfw -q flush
ipfw -q pipe flush

# traffic to bypass shaping
ipfw add skipto 60000 proto tcp src-port 22
ipfw add skipto 60000 proto tcp dst-port 22

# Disable dhcp from crossing the bridge
ipfw add deny ip from any to any bootps via em1

# Static pipes assigned by IP address to the 192.168.201.x subnet
for i in `seq 2 9`
do
  ipfw pipe $i config delay 0ms noerror
  ipfw pipe 30$i config delay 0ms noerror
  ipfw queue $i config pipe $i queue 100 noerror mask dst-port 0xffff
  ipfw queue 30$i config pipe 30$i queue 100 noerror mask src-port 0xffff
  ipfw add queue $i ip from any to 192.168.201.$i out xmit em1
  ipfw add queue 30$i ip from 192.168.201.$i to any out recv em1
done
for i in `seq 10 90`
do
  ipfw pipe $i config delay 0ms noerror
  ipfw pipe 3$i config delay 0ms noerror
  ipfw queue $i config pipe $i queue 100 noerror mask dst-port 0xffff
  ipfw queue 3$i config pipe 3$i queue 100 noerror mask src-port 0xffff
  ipfw add queue $i ip from any to 192.168.201.$i out xmit em1
  ipfw add queue 3$i ip from 192.168.201.$i to any out recv em1
done

ipfw add 60000 pass all from any to any

That mostly creates 2 sets of 89 pipes linking IP addresses in the 192.168.201.2 -> 192.168.201.90 range to pipes for each address. i.e. 192.168.201.10 will use pipe 10 for inbound traffic and pipe 310 for outbound traffic (em1 is the interface connected to the wifi access point so xmit == downlink).

Not sure if it matters or things have changed but the bridge is running FreeBSD 9.2. It looks like 9.3 is still available if you want to be really safe (and if it supports the hardware you are going to run it on).

Once running you also need to install authorized_keys for the agents to connect to the bsd bridge over ssh as root (ssh-copy-id should work from the pi).

Then you statically assign each phone an address in the IP range you selected (192.168.201.x in my case) and configure the pi with the bridge IP and upstream/downstream pipe numbers for that phone's IP address.