Networking setup

[LoyVanBeek]

MSL does this as well, see the last pages of their rule book on http://wiki.robocup.org/images/0/0f/Robocup-msl-rules-2016.pdf We should do the same thing and announce this beforehand.

[komeisugiura]

Nagoya LOC would like to prevent the cases where robots are not working because of the wifi interferences. We will strictly assign the channels, and pay money to an external company to manage this. We also pay money to hire "wifi police" to find and stop unofficial wifi APs including iPhone tethering brought by spectators.

Both Pepper and HSR are strongly dependent on network connections, especially Pepper is required to be connected during its booting sequence.

The details are still under discussion. If we are to use VLAN, the specifications are required to ask the company to estimate the cost. @LoyVanBeek , do you have some ideas on the VLAN specifications?

[LoyVanBeek]

Lets assume 36 teams in total.

• We'll need a VLAN for every team, so the switch/router will need to be capable of handling that many VLANs.
• In total, we'll have a few devices on the wireless network for each team, so the total network needs to be able to handle all those devices. There is also a bit of configuration to do for each, I suppose. If each teams has 1 robot and just 1 laptop in the arena (thus wireless) that is already 72 hosts on the WiFi (over 3 very busy arena's).
• Each arena must be covered by the WiFi acces points. Depending on the size of the arenas, that will be 1 or maybe 2 APs per arena.
• The APs need to be capable of routing robot MAC addresses into their team's VLAN or broadcast separate SSIDs that each map to the VLANs.
• Each VLAN must have internet access
• Reasonable bandwidth and latency between team desk and robot. The faster the better, as always.

[warp1337]

@balkce Reviving this after the German Open, where Tim from TU/e found a nicely working solution for the wireless 'situation'. However, we also discovered/discussed some problems concerning a potential setup for >30 teams.

@LoyVanBeek, can you bring Tim into this discussion?

Noticed problems at the German Open (7 teams in the @Home league):

1) There were no CLEAR rules concerning the general wifi situation (from the beginning). Let's assume we have an official wifi running which more or less reflects the setup described above. The performance of the official setup will significantly drop if teams are also setting up their own wifi's. There are simply not enough channels and thus bandwidth for that. Thus my first proposal: add a rule to the rule book that implements some kind of penalty if teams setup their own wifi. Subsequently, in order to enforce this rule, 'officials' need to screen the team areas for (hidden) wifis and issue warnings and eventually award a penalty. In my experience, the wifi situation will become worse and worse over time if nobody continuously monitors the team area.

2) At the German Open we had two access points providing one SSID and seven sub-nets. The wifi was stable (IMHO) after initial problems but some teams reported internet access problems. I believe this is an issue, but this can only be solved by the local 'provider'

3) If people stream images (e.g. in order to do obejct/person identification), a 640x480 image (e.g. ROS, uncompressed) with a decent framerate will take up to 50-100 mbit/s. So we have >30 teams, all potentially streaming data from their robot 30*100 mbit ... How can this be handled?

4) This issue is related to 3). Assuming we have a physically limited bandwidth, due to the number of wifi channels, access points, or whatever... IMHO, the bandwidth must be divided between the teams STATICALLY. The idea behind this is, if a team is testing stuff (e.g. how fast they can get images) and another team is doing its task, the whole bandwidth is consumed by a team that is not even in the arena. If the bandwidth 'per team' is limited beforehand, and handled by the access points, no problem at all.

Cheers, Florian

[warp1337]

One more addition:

The SSPL wifi should use the 5 GHZ frequency/ies since the Pepper's wifi usb stick (I am not kidding, it's a wifi stick) does not handle 2.4 GHZ very well -> Random disconnects

[HideakiNagano21]

LOC in Japan are trying to find the good policy of WiFi during the robocup. We will monitor all SSID disturbing the team on the task. The detailed information will come soon.

[kyordhel]

@warp1337 Clear rules cannot be defined as each country has different regulations and standards regarding wireless communication and infrastructure, also depending on what is available in the host country and in the venue within the budget.

Komei mentioned that there will be dedicated devices for each league, and that each device can host up to 16 VPN at hardware level. For what I have been told, as long as teams don't set up their own wireless devices, infrastructure will allow all teams to work, limitins the use of the network to robots + 1 device per team member, which seems reasonable to me.

You should contact LOC regarding these tech details.

[komeisugiura]

Nagoya LOC will make a contract with a company to provide the official wifi. It's 5GHz.

We consider three use cases for @Home, where two of them are tested.

1. small setup with HSR robots (tested in March) ->~1k USD
2. medium setup with 7 DSPL teams in Japan Open (tested last week) -> ~10k USD
3. large setup with 34 teams in Nagoya (not yet tested) -> very expensive...

We'll upload the general wifi policy to the Nagoya 2017 website. The details on the @Home-specific issues will be uploaded to the following URL soon. https://github.com/RoboCupAtHome/Nagoya2017

[warp1337]

@komeisugiura Thank you for the clarification. Good work! @kyordhel Sorry, maybe I was a bit fuzzy on the "clear rules" topic. What I meant was not rules relating to the technical infrastructure but rules relating to the teams not allowing them to setup their own wifi. As Komei already said, there will be a general wifi policy available on the web. That's all I wanted :+1:

Regarding the bandwidth limitation, who's in the LOC? That'll be @komeisugiura I guess?

[LoyVanBeek]

Hailing @tue-robotics's @timple into this. At the German Open, during the various team leader meetings, it was discussed that everyone should use the official WiFi that was set up. The SSPL had their own access point even it was only 1 robot, though it ran computations on an external device thus transmitting images over WiFi.

The ideal case that @timple tried to set up was a VLAN for every team but, eventually, the setup was a subnet for every team and each host had to set a static IP. If this is going to be the setup we eventually use, it should be in the rulebook, with subnet allocation per team, the gateway, DNS etc.

There was 1 team that I notified their WiFi access point was up again like 2 times, which I noticed using the WiFi Analyzer app on my phone. In Japan, there must be a penalty for this.

[Timple]

The initial idea of VLANs is a very clean solution where everybody has their own network and cannot access the network of others. Also setting bandwidth limitations is easy on a VLAN. However in Germany the only way I could set up VLANs for every team was to give every team a different SSID (Still same AP and channel, just different name). But this is limited to 4 SSIDs per channel and therefor does not scale to 36 teams.

Another implementation is using a radius server. There you can couple each user to a VLAN. I do not have experience with this nor did we have the hardware for this at the German Open.

Thus the solution implemented at the German Open was without VLANs. The situation was as follows: There is one @Home network consisting of the two channels provided by the organization. Every access point broadcasts the same SSID, dividing users over the access points was done by the access points themselves. This should be more optimal than letting users chose one.

Everybody on the network has its own subnet:

Team name	IP address range
b-it-bots	192.168.41.*
RT Lions	192.168.42.*
SCC	192.168.43.*
TU/e	192.168.44.*
home	192.168.45.*
ToBI	192.168.46.*
SocRob	192.168.47.*
SSPL – Teams	192.168.48.*
DSPL – Teams	192.168.49.*

(Plenty of subnets left, so just extend the team list with other teams).

Furthermore everybody is allowed to be on the 192.168.1. range to the 192.168.20. range.

On the German Open the 192.168.11.* range was the DHCP range. This is useful for people unable or unwilling to set a fixed IP.

All in all the setup was very stable after 2-days of configuring it. The last three days there was over 400GB send or received over the Wifi. Wireless roundtrip times (ping) to our robot on wifi kept around 20ms. Of course we connected our laptops to a cable to save half the wifi bandwidth. This increases performance for us and keeps wifi usage to a minimum.

I do not see a problem in 'forcing' people to use their laptops on a cable. The organisation can supply a switch near every arena because everybody is on the same network anyway.

I also would strongly recommend a main switch capable of detecting loops and dhcp servers in the network and blocking them. This will prevent the network shutting down every time a team makes a mistake on the cabling.

[warp1337]

@Timple +1 for the cable and info!

[kyordhel]

Rulebook & penalties.

He rulebook must clearly state penalties for broadcasting wireless, unintentionally drop the network, and intentionally hacking stuff. All other distributions (IP tables and other more logistical stuff) should not go in the rulebook but in a page, as Komei pointed out.

Something that can also be limited is the use of WiFi devices within the arenas and team area. For what I know, a standard @Homer often comes with 3 devices: laptop, cellphone, and tablet, that for some mysterious reason need Internet connection or the owner can lose their sanity. A clear rule may make mandatory the use of wired infrastructure and restrict mobile devices to the use of 3G. Penalty: confiscation.

Radius & Networking

Doable with a good server and an amazing switches and access points. I've set up radius in the past with MAC filtering and is pretty easy to assign each group to a different segment of the network isolated from the others (although it was in lab conditions for ~50 devices). Without encryption, performance is much better but you still need a powerful, very powerful server. On the other hand, there are devices that do that natively at hardware level with VPN, so those might be the ones being used in Komei's tests.

One thing is for sure: all that networking hardware is really expensive, specially if it can block other DHCP and cycles.

Faith in humanity

Is lost beyond redemption. Teams will do whatever it comes to have access to the network in an easy way because they need to win, fix stuff, and they don't have time for dealing with networks they are not familiar with. @balkce and I have the bad experience of catching teams actively cheating during a test. I am afraid they will do whatever it comes to in order to win. Maybe to Europeans and Japanese people this is unthinkable, but in other cultures such behavior is even encouraged.

Undisclosed information

Could you please provide the name of that team, @LoyVanBeek. They deserve a friendly warning (specially if the wireless is used internally by the robot).

[rventura]

I think that (1) the wifi network having sufficiently good performance and (2) being trivial to setup (and maintain) are two preconditions for the fairness of stringent wifi rules. Otherwise, teams will tend to (regrettably) cheat, since poor wifi will get into the way of their work. Unfortunately I was unable to attend German Open (due to my newborn daughter), but I’m happy to know it worked fine. I was told that MSL league has no wifi issues, and I remember that IP subnet’ing was being used in the past.

@komeisugiura: one question, did you mean 5GHz only? That is, no 2.4GHz?

Best, Rodrigo Ventura

[komeisugiura]

The draft version of the wifi policy can be found here. This shows the general rules for the event, not only @home. RoboCup2017_WirelessGuideline_03(20170425)_extracted.pdf

In Nagoya,

• The channels will be strictly controlled.
• We will have "wifi police" who detect unofficial APs.
• The monitored wifi situation will be shown realtime on a large screen. Team members can see the noise level

[Timple]

The cheapest switch I could find having these features and allowing 36 teams is the Ubiquiti UniFi Switch 48. But this is of course up to the LOC.

Would it be an idea to have a single channel free-for-all-participants wifi over the whole venue? I notice the urge of people to have whatsapp etc working on their phone. For communication to home etc. I detected a lot of android and iphones on the wifi network.

I am quite sure that if a (even very slow) alternative would be present in the halls, people would not feel the need to use the arena wifis for this.

[LoyVanBeek]

@timple +1 on that, just 1 channel perhaps indeed throttled. Would be good for social media attention as well.

@komeisugiura that is the most detailed WiFi guideline I've ever seen, very nice.

[komeisugiura]

The followings are the network configurations and traffic statistics in RoboCup@Home JapanOpen 2017. Due to the budget limitation, the official network was provided only for DSPL, not for OPL/SSPL.

May 3, 2017: TrafficReport_Robocup Japan Open_2017-05-03.pptx May 4, 2017: TrafficReport_Robocup Japan Open_2017-05-04.pptx May 5, 2017: TrafficReport_Robocup Japan Open_2017-05-05.pptx

In Nagoya 2017,

• Of course OPL and SSPL will be provided with the official wifi as well
• Each AP can handle up to 16 VLANs by a single device, which is enough for 16 teams in OPL (=the largest among @home leagues)
• Each of the 34 teams will have a unique SSID

[LoyVanBeek]

Hi @komeisugiura that is awesome. Assuming you mean VLANs instead of VPNs, this is indeed the setup as described by the rulebook.

One question: how are AP devices distributed over arena's? We're going to have 3 arenas and 3 leagues. Is the schedule such that each league only plays in 1 arena an thus a team's SSID is only broadcasted in the team's arena?

[komeisugiura]

@LoyVanBeek thanks! Yes I meant VLANs.

Each team's SSID will only be broadcasted in the corresponding league's arena. Access point roaming is under consideration. Currently I'm considering how to provide networking in the Restaurant area and the arena for the Final, where teams from SSPL/DSPL/OPL use the same official wifi.

[warp1337]

Just as a comment: when I re-spawned this thread I didn't expect that much response. Awesome work everyone! That's the spirit.

[warp1337]

@komeisugiura Can you make an educated guess how much bandwidth can be guaranteed (based-on your tests) by the current setup per team? 100 mbit/s, 10 mbit/s, 1 mbit/s.

Thank you 🥇

[kyordhel]

I don't think any other change needs to be done to this year's Rulebook regarding the network. Therefore I will close this issue (although the discussion can continue here).

[warp1337]

@kyordhel You read the above right? ;) I wasn't asking about the internet bandwidth... However, thanks for your kind answer.